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ABSTRACT 

Reading Recovery (RR) is a short-term intervention for the 
lowest-achieving children in first grade. Children meet individually with a 
specially trained teacher for 30 minutes daily for 12 to 20 weeks. There is 
substantial scientific evidence to support Reading Recovery’s effectiveness 
with the lowest-performing first-grade students. Reading Recovery does not 
claim to be the only solution to the nation’s reading problems, nor does it 
seek preferential treatment for funding under the No Child Left Behind Act. 
Rather, RR seeks the right to be considered as an early intervention option 
for state and local educational authorities. This report is divided into the 
following sections: Executive Summary; Introduction; Section I: Reading 
Recovery is highly successful with the lowest-performing first-grade classes 
(Research demonstrates RR effectiveness; Evaluation evidence supports RR 
effectiveness; Bias in the Internet letter is analyzed; Summary); Section II: 
Cost-effectiveness is a complex concept in education (Cost-effectiveness 
cannot be oversimplified; One-to-one interventions are essential for 
accelerated learning of the lowest performing first graders; Research cited 
by the Internet letter is flawed; Summary); Section III: Reading Recovery 
uses standard assessment measures (RR uses an observation survey of early 
literacy achievement; RR students perform well on norm-referenced tests; 
Summary); and Section IV: Change is an integral part of the Reading Recovery 
design (RR has built-in mechanisms for change; RR responds to Internet letter 
change recommendations; Evidence shows that RR teaches phonemic awareness and 
phonics; New Zealand researcher responds to the Internet letter; Summary) . 

The Internet letter, a list of university training centers, and a response 
letter are appended. (Contains 71 references.) (NKA) 
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EXECUTIVE SUMAMRY 



In May 2002, a letter criticizing Reading 
Recovery was widely distributed to mem- 
bers of Congress and the education com- 
munity via the Internet. Although the let- 
ter purports to be an academic debate, its 
motivation appears to be political. The let- 
ter was released as states and local school 
districts were developing budgets and fed- 
eral grant applications. The letter builds a 
distorted case based on flawed research and 
selective reporting of Reading Recovery 
studies (Appendix A). 

In the national debate about scientifically 
based research and accountability, Reading 
Recovery is a surprising target because no 
program is more accountable and has a 
stronger scientific base than Reading 
Recovery. Reading Recovery is a short-term 
intervention for the lowest-achieving chil- 
dren in first grade. Children meet individ- 
ually with a specially trained teacher for 
30 minutes daily for 12 to 20 weeks. 
Children are tested before entering 
Reading Recovery to assure that they are 
the lowest-achieving readers in their class. 
They are also tested after their lessons are 
discontinued and at the end of first grade. 
The outcome of their lessons is compared 
with a random sample of their peers. 
Results are reported on school, district, and 
national levels. 

Cumulative 17-year results show that in 
the United States, 60% of all children 
served can read at class average after their 
lessons, and 81% of children who have the 
full series of lessons can read at class aver- 
age. No other intervention in the United 
States has such an extensive database and 
such strong accountability. More than one 
million children have been served in 
Reading Recovery since it came to the 
United States in 1984 through a team of 
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In Reading Recovery, children meet individually with 
a specially trained teacher for 30 minutes daily for 
1 2 to 20 weeks. 



researchers at The Ohio State University. 
Reading Recovery's not-for-profit network 
connects 23 universities, 3,293 school dis- 
tricts, Department of Defense Schools, and 
10,622 elementary schools in the United 
States alone (National Data Evaluation 
Center [NDEC], 2002; see Appendix B for 
a complete list of Reading Recovery uni- 
versity training centers). Reading Recovery 
is not only available for children struggling 
to learn to read in English: it has been 
reconstructed in Spanish, French, and 
Maori and is currently being reconstructed 
in other languages. 

The Internet letter chooses to ignore all of 
this easily available information in an 
attempt to undermine public confidence in 
Reading Recovery. In addition, the 
Internet letter reflects a broader public 
debate about the nature of scientific evi- 
dence in reading research and the relation- 
ship of federal policy to local school deci- 
sion making. These issues are discussed 
more fully in a response letter signed by 
more than 200 academic leaders and 
researchers outside Reading Recovery. The 
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signers represent an international group of 
independent scholars and researchers who 
have studied language, literacy, and learn- 
ing in many contexts (Appendix C). This 
letter, entitled “A Broader View of 
Evidence: Reading Recovery as an 
Example,” makes the following key points: 

• Public education dollars belong 
to citizens, not to a small group 
of researchers who have a particu- 
lar point of view. 

• A scientific stance requires a com- 
plete, evidence-based analysis of 
any educational program. 

• Policy makers have the responsi- 
bility to consider evidence from a 
wide range of perspectives and 
validated research models. 

• Responsibly and rigorously col- 
lected evaluation data provide 
legitimate evidence of program 
success. 

• An early intervention program 
like Reading Recovery is part of a 
comprehensive literacy effort. 

This report is written by Reading Recovery 
researchers and academics from Reading 
Recovery’s university training centers in the 
United States and is a response to the four 
criticisms in the Internet letter. 

1. Reading Recovery is highly 
successful with the lowest- 
performing first-grade students. 

Research in peer-reviewed journals docu- 
ments Reading Recovery’s effectiveness 
(Center, Wheldall, Freeman, Outhred, & 
McNaught, 1995; Iversen &Tunmer, 

1993; Pinnell, 1997; Pinnell, Lyons, 
DeFord, Bryk, & Seltzer, 1994; Sylva & 
Hurry, 1996; Wasik & Slavin, 1993). 
Objective critics acknowledge that Reading 
Recovery works: “Evidence firmly supports 
the conclusion that Reading Recovery does 



bring the learning of many children up to 

that of their average-achieving peers It is 

clear that many children leave the program 
with well-developed reading strategies, 
including phonemic awareness and knowl- 
edge of spelling” (Shanahan & Barr, 1995, 
p. 989). 

Many evaluation studies demonstrate that 
the majority of Reading Recovery students 
maintain and improve their gains in later 
grades. Several studies using widely accept- 
ed standardized measures or state assess- 
ment measures show strong results for 
Reading Recovery students (Askew et al., 
2002; Brown, Denton, Kelly, & Neal, 
1999; Pinnell, 1989; Rowe, 1995; Schmitt 
& Gregory, 2001). 

Former Reading Recovery students, like all 
students, need good classroom teaching to 
continue their progress. Reading Recovery 
is a short-term safety net, an essential com- 
ponent in a school’s comprehensive literacy 
program. Two studies in refereed journals 
reveal that Reading Recovery students 
experience gains in self-concept (Cohen, 
McDonnell, & Osborn, 1989; Rumbaugh 
& Brown, 2000). 

Evidence supporting Reading Recovery’s 
effectiveness not only appears in peer- 
reviewed journals. It is also evident in the 
evaluation data collected and reported 
annually by the National Data Evaluation 
Center located in the College of Education 
at The Ohio State University. The data 
allow local administrators and school 
boards to monitor children’s results and to 
examine implementation data such as the 
number of lessons missed, reasons for 
missed lessons, and level of implementa- 
tion in a school. 

To advance their political agenda, the 
authors of the Internet letter selectively 
report and distort the limited set of experi- 
mental studies they present to their col- 
leagues, politicians, and the public. The 
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clearest instance of this distortion is in 
their use of the Elbaum, Vaughn, and 
Moody (2000) meta-analysis. The letter 
states, “In fact, for the poorest readers, 
empirical synthesis of ‘in-house’ and inde- 
pendent studies indicates that Reading 
Recovery is not effective. In Elbaum et al. 
(2000), the gains for the poorest readers 
instructed by Reading Recovery were 
almost zero” (paragraph 3). 

Why is this a distortion? It ignores the 
major finding of this meta-analysis that the 
effect of Reading Recovery on student per- 
formance was large and significant. The 
Elbaum et al. (2000) study states, “For 
Reading Recovery interventions, effects for 
students identified as discontinued were 
substantial, whereas effects for students 
identified as not discontinued were not sig- 
nificantly different from zero” (p. 605). 

The Internet letter emphasizes the small 
number of students who did not make 
progress while it ignores the fact that the 
majority of students made substantial 
progress. It also implies that the not- 
discontinued students were the ones with 
the lowest entry scores. This is not true. 
The interpretation of this information in 
the Internet letter seems biased. 



2. Cost-effectiveness is a complex 
concept in education. 

The Internet letter states that “Reading 
Recovery is not cost-effective because the 
developers require one-to-one interventions 
by highly trained teachers” (paragraph 4). 

It is inappropriate to label a program as 
expensive or not cost-effective without 
extensive research comparison with other 
programs that target the same student 
population and seek to achieve the same 
results. Both long- and short-term benefits 
must be considered in this type of research. 
Such studies are rare in medical research 
and almost nonexistent in educational 
studies. 



By intervening early, Reading Recovery 
reduces referrals and placements in special 
education (NDEC, 2002), limits retention, 
and has demonstrated lasting effects. 
Retention and special education referral 
each have a substantial price tag. The local 
costs of providing Reading Recovery servic- 
es for 12 to 20 weeks are substantially less 
than special education and retention costs, 
particularly when the majority of Reading 
Recovery children sustain and improve 
their literacy learning gains in subsequent 
years of primary education (Brown et al., 
1999; Schmitt & Gregory, 2001). 

Authors of the Internet letter base their 
recommendations on inadequate research. 
The 2000 meta-analysis by Elbaum, 
Vaughn, and Moody is again cited. A clos- 
er look at the meta-analysis reveals that evi- 
dence is based on an unpublished doctoral 



By intervening early, Reading Recovery reduces 
referrals and placements in special education. 



dissertation (Evans, 1996) and an unpub- 
lished master’s thesis (Acalin, 1995). Evans’ 
doctoral dissertation supporting the effec- 
tiveness of group instruction is based on a 
very small sample of eight children: four 
randomly assigned to Reading Recovery 
and four assigned to a small group inter- 
vention. The Reading Recovery teacher 
studied in the dissertation was in the first 
months of the training year and had not 
recently taught primary-grade students. 
Evidence of equivalence for the two groups 
was lacking at pre-test. 

Evidence from the Acalin master’s thesis is 
even more suspect. Reading Recovery les- 
sons were not even delivered by a Reading 
Recovery teacher, but by special education 
teachers who had not participated in 
Reading Recovery training. Furthermore, 
although Reading Recovery is a first-grade 
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intervention, Acalin provided instruction 
to 66 subjects in first through fourth 
grades. Only eight of the children were in 
first grade, with four assigned to Reading 
Recovery and four to Project Read. 

In addition to the two studies in Elbaum, 
the Internet letter cites another unpub- 
lished doctoral dissertation (Iversen, 1997) 
as support for group intervention. Iversen 
claims to compare Reading Recovery with 
an instructional intervention for groups of 
two. The Reading Recovery program, how- 
ever, was not standard with regard to train- 
ing, screening and selection procedures, or 
teaching procedures. Design and method- 
ological issues also raise numerous ques- 
tions. Therefore, it would be inappropriate 
to draw any conclusions about Reading 
Recovery from the data presented. 

Even casual consumers of scientific 
research would wonder why the Evans and 
Acalin studies were considered to have met 
criteria for inclusion in a meta-analysis that 
purports to follow, in the authors’ own 
words, “best practices for research synthe- 
sis” (Elbaum et al., 2000, p. 606). 

Studies cited by Elbaum and her colleagues 
provide virtually no evidence to support a 
change from one-to-one to small group 
instruction for the lowest-achieving first 
graders. The suggestions for Reading 
Recovery to change from one-to-one 
instruction is especially weak because there 
is documented evidence of success with 
hundreds of thousands of the lowest- 
performing first-grade students. Reading 
Recovery has also developed a design for 
dissemination and teacher training that 
allow these results to be replicated in 
diverse contexts across the United States 
and the world. 

One-to-one tutoring is a central aspect of 
both the theory and design of Reading 
Recovery, and there is a body of research to 
support it. Extensive research would be 
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needed to demonstrate the implications of 
a change from individual to small group 
instruction; however, researchers within 
and outside Reading Recovery should con- 
tinue to study all possibilities. Research 
supports one-to-one tutoring and indicates 
that it may be essential for children who 
are at high risk (Bloom, 1984; Juel, 1991; 
Wasik & Slavin, 1993). The systematic 
nature of Reading Recovery instruction is 
based on a teachers detailed assessment 
and analysis of a child’s knowledge base 
and skills. The teaching is highly efficient 
because the teacher has this precise inven- 
tory of skills and strategies and is able to 
teach exactly what the child needs to know 
next. 

3. Reading Recovery uses standard 
assessment measures. 

Reading Recovery pre-tests and post-tests 
students using the measures published in 
An Observation of Early Literacy Achieve- 
ment (Clay, 1993a/2002). The survey is a 
standard set of measures developed in 
research studies with qualities of sound 
assessment instruments having reliability, 
validity, and discrimination indices. It was 
developed to meet the unique need to 
assess emergent literacy in young children. 
The survey is comprised of six literacy 
tasks with established validity and reliabili- 
ty: letter identification, word test, concepts 
about print, writing vocabulary, hearing 
and recording sounds in words, and text 
reading. The Internet letter suggests a pref- 
erence for norm-referenced tests that are 
widely available and commonly used in 
reading intervention research. Although 
these tests may yield valid comparisons for 
students who are already reading, they are 
not sensitive to variability in emerging 
knowledge and are not useful as baseline 
measures to assess change in early literacy. 
Some studies, however, have used stan- 
dardized measures and state assessments to 
explore subsequent performance of former 
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In Reading Recovery, children are shown how to use 
letter-sound relationships to solve words in reading 
and writing and how to use structural analysis of 
words and to learn spelling patterns. 

Reading Recovery children (Askew et al., 
2002; Brown et al., 1999; Schmitt & 
Gregory, 2001); they show that Reading 
Recovery children do in fact maintain and 
improve their gains. 

4. Change is an integral part of the 
Reading Recovery design, 

Reading Recovery has built-in mechanisms 
for change. The central program document 
is Reading Recovery: A Guidebook for 
Teachers in Training (Clay, 1993b). 
Originally published as Early Detection of 
Reading Difficulties (1979/1985), it was 
thoroughly revised and retitled in 1993. 
The Guidebook reveals significant additions 
over years of development including: 

• more intensive attention to and 
detailed description of the role of 
phonemic awareness; 

• explicit directions for teachers in 
helping children use letter-sound 
relationships and phonics; 

• more deliberate focus on compre- 
hension strategies during the 
reading of a new book; 



• differentiation between the way 
the teacher supports children dur- 
ing the reading of a new text and 
the role of familiar reading; and 

• more information on how to 
teach for fluency and phrasing. 

In addition to changes reflected in the 
Guidebook, Reading Recovery uses results 
from the National Data Evaluation Center 
to analyze and guide changes in implemen- 
tation at the local, state, and national lev- 
els. Reading Recovery’s extensive and con- 
tinuing training for all personnel supports 
changes. Change is also evident in the 
standards and guidelines of the Reading 
Recovery Council of North America. All 
schools must adhere to these standards in 
order to be in compliance with the royalty- 
free trademark granted annually to partici- 
pating schools by The Ohio State Univer- 
sity. This trademark helps to ensure quality 
and consistency in Reading Recovery. 
Reading Recovery is a not-for-profit part- 
nership of universities and local school 
districts. 

The Internet letter recommends explicit 
instruction in phonics and phonemic 
awareness and suggests that Reading 
Recovery ignores these important instruc- 
tional components. Any astute observer of 
a Reading Recovery lesson would recognize 
the explicit teaching of letters, sounds, and 
words. In Reading Recovery, children are 
shown how to use letter-sound relation- 
ships to solve words in reading and writing 
and how to use structural analysis of words 
and to learn spelling patterns (Pinnell, 
2000). A study by Stahl, Stahl, and 
McKenna (1999) demonstrates that 
Reading Recovery students do in fact per- 
form well on standardized tests of phone- 
mic awareness and phonological coding. 

One academic researcher studying a wide 
range of programs made the following 
observation about Reading Recovery: ‘‘The 
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importance of phonological and linguistic 
awareness is also explicitly recognized” 
(Adams, 1990, p. 420) and went on to 
describe Reading Recovery as one of several 
programs that “are designed to develop 

thorough appreciation of phonics On 

the other hand, none of these programs 
treats phonics in a vacuum” (p. 421). 

The Internet letter cites two studies to sup- 
port its contention that Reading Recovery 
would improve with more explicit phonics. 
The first, a 1993 study by Iversen and 
Tunmer, recommended that Reading 
Recovery add explicit phonics and phone- 
mic awareness. In fact, the modifications 
had already been made before this study 
was carried out. Iversen herself had been 
trained as a teacher leader in the early years 
of Reading Recovery training in New 



One academic researcher studying a wide range 
of programs made the following observation 
about Reading Recovery: “The importance of 
phonological and linguistic awareness is also 
explicitly recognized/’ (AdamSy 1990) 



Zealand, but at the time of the study, she 
was no longer teaching Reading Recovery 
and was not attending continuing profes- 
sional development to receive program 
updates. Thus, the Reading Recovery 
instruction provided in the study was out 
of date, even in 1993. Despite this short- 
coming, results of the Iversen and Tunmer 
study indicated that both the traditional 
Reading Recovery group and the phonics- 
enhanced group outperformed a control 
group on all measures of phoneme deletion 
and phoneme segmentation. The group 
with increased emphasis on phonemic 
awareness had programs that were shorter 
than Iversens traditional group. Interest- 
ingly, the increase in phonemic awareness 
and phonics that Iversen included in train- 
ing of her experimental group had already 



been included in Reading Recovery pro- 
grams around the world. 

The second study cited in the Internet let- 
ter in support of explicit phonics is by 
Morris, Tyner, and Perney (2000). This 
study looked at some alternative staffing, 
training, and instructional approaches to 
early intervention. Morris and his col- 
leagues did place a greater emphasis on iso- 
lated word study, but they also modeled 
more than three-quarters of the lesson for- 
mat on Reading Recovery. The results indi- 
cated that students who participated in 
their First Steps program made better 
progress than a matched group of low stu- 
dents in non-participating comparison 
schools. This study was not designed to 
compare results against Reading Recovery 
or to isolate the contribution of a particu- 
lar form of word study in relation to other 
program components. The claim in the 
Internet letter that “the addition of an 
explicit component addressing spelling-to- 
sound patterns was highly effective” (para- 
graph 7) seems questionable given that 
First Steps students received tutoring for 
the entire school year, averaging 91 lessons 
per student. 

In the debate about how to teach phonics, 
the authors of the Internet letter draw on a 
1999 report of the Literacy Experts Group 
in New Zealand. The group recommended 
“greater emphasis on explicit instruction in 
phonological awareness and the use of 
spelling-to-sound patterns in recognizing 
unfamiliar words in text” (as quoted in 
Internet letter, paragraph 9). In response to 
the Internet letter and this recommenda- 
tion, a member of the Literacy Experts 
Group wrote: 

It would be regrettable if. . . [any of 
the] recommendations from the 
1999 Literacy Experts Group was 
construed as meaning that this 
group was among those attacking 
Reading Recovery. Because it 
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wasnt...Most striking however, is 
the clear message that most of this 
debate is about some researchers 
talking to some other researchers, 
with very little buy-in from the 
teachers who implement Reading 
Recovery, or those with experience 
of translating research findings into 
effective classroom programmes. 

And, there is a huge gulf between a 
research study and a programme 
that works in a classroom. This is 
one area where the four New 
Zealand signatories of the U.S. 
[Internet] letter are out-of-step 
with many other reading 
researchers and literacy educators 
in New Zealand. (Croft, 2002, 
pp. 2-3) 

In summary, there is substantial scientific 
evidence to support Reading Recovery's 
effectiveness with lowest-performing first- 
grade students. Reading Recovery does not 
claim to be the only solution to the 
nations reading problems, nor does it seek 
preferential treatment for funding under 
the No Child Left Behind Act. Rather, 
Reading Recovery seeks the right to be 
considered as an early intervention option 
for state and local educational authorities. 
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A. Background on the Internet letter 

In May 2002, a letter criticizing Reading Recovery was widely dis- 
tributed to members of Congress and the education community 
via the Internet. The letter identified the signers as “an interna- 
tional group of researchers who study reading development and 
interventions with struggling readers” (paragraph 1; see Appendix 
A for the complete text of the letter). 

Although the Internet letter purports to be an academic debate, its 
motivation appears to be political. The letter was released as states 
and local school districts were developing budgets and federal 
grant applications. 

While the signers of the Internet letter say their goal is “not to dis- 
credit Reading Recovery” (paragraph 2), it appears that Reading 
Recovery was singled out because in February 2002, local school 
supporters had effectively advocated to change draft Guidance 
issued by the U.S. Department of Education. The draft Guidance 
contradicted the statute (PL 107-1 10, No Child Left Behind Act 
of 2001) and undermined local control over educational decisions. 
Among other provisions, the draft Guidance 

• limited use of funds to classroom instruction only 

• required 90 minutes of uninterrupted instructional time 
for reading lessons 

• omitted use of funds for professional development in 
early intervention 

At a Senate hearing in April 2002 for the No Child Left Behind 
Act, three Senators spoke in favor of local control and specifically 
inquired about Reading Recovery's eligibility. It is important to 
note that Reading Recovery has never asked for preferential treat- 
ment in the use of public funds. Rather, Reading Recovery sought 
the right to be considered as an early intervention option for state 
and local educational authorities. 

In a 1995 independent evaluation of Reading Recovery, Shanahan 
and Barr argued for local choice: 

It is appropriate, in our opinion, to continue to expend 
public funds in support of Reading Recovery. It would be 
wrong to accept it as the only appropriate intervention for 
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“It is appropriate, in our 
opinion, to continue to 
expend public funds in 
support of Reading 
Recovery. It would be 
wrong to accept it 
as the only appropriate 
intervention for children at 
risk. Public policy should 
permit local education 
agencies to adopt Reading 
Recovery or other proven 
approaches, and should 
encourage local 
experimentation and 
innovation to identify even 
better approaches.” 

(Shanahan and Barr, 1995) 
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Reading Recovery is now 
in about 20% of public 
elementary schools with 
first-grade classrooms, 
and it works well with a 
variety of good classroom 
literacy programs. 



children at risk. Public policy should permit local educa- 
tion agencies to adopt Reading Recovery or other proven 
approaches, and should encourage local experimentation 
and innovation to identify even better approaches, (p. 992) 

Reading Recovery is now in about 20% of public elementary 
schools with first-grade classrooms, and it works well with a vari- 
ety of good classroom literacy programs. As one part of a school s 
comprehensive literacy program, Reading Recovery does not claim 
to be the solution for all students. The Reading Recovery commu- 
nity believes there are many paths to literacy and that no single 
program will make readers of all children. What Reading Recovery 
does provide is the most expert teaching for the most vulnerable, 
low-performing first-grade students. Ajid, it provides a dynamic 
model of professional development. 

The question has been asked: Why is the Internet letter considered 
a political attack and riot an academic debate? The answer lies in 
the tone of the letter, in the choice of language, and in the lack of 
balance and civility that characterizes true academic exchange. 

The Internet letter begins by stating, “the findings here are sum- 
maries of several peer-reviewed studies and syntheses of research 
on Reading Recovery. However, it is not our goal to discredit 
Reading Recovery, but as with any other program, outline its 
weaknesses to suggest how it can be improved” (paragraph 2). 

Unfortunately, after this reasoned opening, the letter drops any 
pretense of balance by refusing to acknowledge Reading Recovery’s 
documented success with hundreds of thousands of lowest- 
performing beginning readers in the United States, as well as 
thousands more in New Zealand, Australia, Canada, and the 
United Kingdom. 

The Internet letter has a number of biases and omits important 
findings. Among them, 

• The letter fails to mention the experimental random- 
assignment study funded by the MacArthur Foundation 
establishing Reading Recovery effectiveness (Pinnell et 
al., 1994). While the authors of the Internet letter might 
argue that the MacArthur Foundation researchers were 
supporters of Reading Recovery, the world of peer- 
reviewed articles would be quite limited if researchers 
who were testing programs did not publish articles 
about their results. In fact, external research review 
was built in at the design and audit of data. The 
MacArthur Foundation appointed a review team of liter- 
acy experts from outside the Reading Recovery commu- 
nity. In addition, a University of Chicago team provided 
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input into the study design and conducted a separate and 
independent audit of the data. The report of the study 
was published in the premiere peer-reviewed journal of 
reading research, Reading Research Quarterly, and met all 
criteria to be included in the National Reading Panel 
report (2000). 



• The letter cites two unpublished doctoral dissertations 
(Evans, 1996; Iversen, 1997) as evidence that group 
interventions are as effective as one-to-one interventions. 
Yet these studies provide weak support for their argu- 
ment. The Evans study included a very small sample, just 
four children who received lessons from a teacher who 
was in her first months of Reading Recovery training. 

The Iversen dissertation did not use groups, but 
matched pairs of students. At the same time, the 
Internet letter ignores studies by researchers that 
document the effectiveness of one-to-one interven- 
tions for lowest-performing students (Dorn & Allen, 
1995; Pinnell et al., 1994; Wasik & Slavin, 1993). 

• The letter bases conclusions on a meta-analysis by 
Elbaum et al. (2000) that ignored one of its own 
major findings: that Reading Recovery had a signifi- 
cant effect on results (d = 0.66; p. 615). Elbaum 
and her colleagues also included studies that do not 
meet criteria for inclusion in a meta-analysis (for 
example, Evans, 1996; Acalin, 1995). The insuffi- 
cient parameters in the Evans study have already 
been discussed. The Acalin study included 66 
treatment subjects, only eight of whom were in first 
grade, and of those eight, four were in Reading 
Recovery. Treatment descriptions provided in the 
Acalin study reveal wide discrepancies from standard 
Reading Recovery procedures. 




Reading Recovery serves as a safety net providing 
one-to-one instruction for first-grade children having 
difficulty with literacy. 



• The letter charges that Reading Recovery does not 
report data on all children, even though data and the 
explanation of procedures are publicly available from the 
National Data Evaluation Center Web site verifying that 
data are reported on every child who ever enrolled in 
Reading Recovery. 



Clearly, the Internet letter is not an unbiased review of evidence. 

It represents a narrow but vocal minority opinion. In response, 
researchers and practitioners prepared a letter entitled “A Broader 
View of the Evidence: Reading Recovery as an Example” (see 
Appendix C). Within four weeks of its initial circulation, over 200 
reading researchers and educators from outside Reading Recovery 
had signed the letter honoring a broader view of research and 
using Reading Recovery as an example. 
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Following the release of the response letter, Reading Recovery 
researchers and practitioners prepared this more detailed report, 
“What Evidence Says About Reading Recovery.” Before reviewing 
the four charges in the Internet letter, the following section pro- 
vides a brief overview of Reading Recovery and the role it plays as 
a part of a schools comprehensive literacy program. 



Reading Recovery is a 
short-term intervention 
that supports classroom 
instruction for the lowest- 
achieving children in first 
grade. Children meet 
individually with a specially 
trained teacher for 30 
minutes daily for 1 2 to 
20 weeks. 
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B. Reading Recovery overview 

Reading Recovery is a short-term intervention that supports class- 
room instruction for the lowest-achieving children in first grade. 
Children meet individually with a specially trained teacher for 30 
minutes daily for 12 to 20 weeks. Reading Recovery serves as a 
safety net for children having difficulty with literacy learning in 
any good classroom program. Results indicate that Reading 
Recovery meets the challenge of closing the gap early, before a 
cycle of failure begins. Components contributing to Reading 
Recovery’s effectiveness within a school system are teaching, pro- 
fessional development, evaluation and accountability, and imple- 
mentation in schools. These components are briefly described in 
the following sections. 

Teaching 

Reading Recovery supplies one-to-one instruction to the most at- 
risk children from highly skilled teachers. Through Reading 
Recovery professional development, these teachers develop an 
understanding of literacy processes and literacy acquisition. They 
learn how to observe children closely and decide when it is most 
effective to introduce new learnings as well as when and how to 
scaffold learning support. Reading Recovery “brings hardest-to- 
teach children to a level where they can be full participants in the 
classroom program” (Clay, 1993b, precedes p. 1, paragraph 2). 

The lowest-performing children need individually designed lessons 
that follow each child’s unique path to literacy learning. Lessons 
include explicit attention to the five essential components of read- 
ing instruction: phonics, phonemic awareness, fluency, compre- 
hension, and vocabulary. “To get results with the lowest achievers 
the teacher must work with the particular (and very limited) 
response repertoire of a particular child using what he knows as 
the context within which to introduce him to novel things” (Clay, 
1993b, p. 8). One-to-one tutoring has repeatedly been found to 
be the most effective approach to prevent reading failure (Dorn & 
Allen, 1995; Pinnell et ah, 1994; Slavin, Karweit, & Wasik, 1992). 

Reading Recovery epitomizes the model of prevention of reading 
failure as articulated by Pianta (1990). It represents a secondary 
prevention strategy for children who do not respond fully to a pri- 
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mary prevention strategy such as good classroom teaching. The 
effectiveness of Reading Recovery for struggling readers in the 
context of a comprehensive school approach to prevention is well 
documented (see Allington & Walmsley 1995; Crevola & Hill, 
1998; Pikulski, 1994; Wasik &C Slavin, 1993). 

Professional development 

Extensive training and ongoing professional development are hall- 
marks of Reading Recovery and are the reasons that Reading 
Recovery teachers can bring 60% of all lowest-performing first- 
grade children served up to class average in just 12 to 20 weeks of 
individual lessons. Reading Recovery has a three-tiered structure of 
university training centers, training sites, and schools to support 
teacher training and implementation. 

In the United States, 23 university training centers provide the 
yearlong graduate program to train Reading Recovery teacher lead- 
ers^ (Appendix B). The university trainers are faculty members at 
these centers. They teach children, engage in research, support 
implementation at affiliated teacher training sites, and provide 
ongoing professional development for Reading Recovery personnel 
at all levels. 

At the training site level, 723 teacher leaders provide yearlong ini- 
tial training for Reading Recovery teachers who receive university 
credit. Teachers not only attend weekly classes to discuss theory 
and practice, but they also apply their learning every day through 
teaching Reading Recovery children. 

Reading Recovery teachers at 10,622 elementary schools are sup- 
ported by their teacher leaders. Teacher leaders visit schools to 
observe lessons and consult on hard-to-teach children. They also 
provide ongoing professional development for trained Reading 
Recovery teachers to continue building teaching skills and to pro- 
vide information about changes in Reading Recovery. 

Evaluation and accountability 

Evaluation data that are responsibly and rigorously collected pro- 
vide legitimate and strong evidence of program success. Each 
Reading Recovery school is required to report its data to a national 
center annually in exchange for royalty-free use of the trademark. 
Data reporting is used for decision making and is integral to quali- 
ty assurance. 

The evaluation methodology is standardized nationally and follows 
accepted principles of evaluation research. Methodology docu- 



• For the Reading Recovery overview discussion, figures given for staff levels reflect those 
of the 2000—200 1 school year. 
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meats and research reports are available on NDEC s Web site 
(http://ndec.reading-recovery.org). 

Implementation in schools 

Positive results for Reading Recovery students depend not only on 
instruction, but also upon a school environment that supports effi- 
cient operations. Among the factors that affect results are 

• daily lessons for students 

• scheduling for students and teachers 

• personnel to supply adequate teaching time 

• collaboration with classroom teachers 

• teacher selection 

• adequate space and materials 

• administrative support 

• shared ownership and understandings in the school 
and community 



C. Organization of “What Evidence Says About 
Reading Recovery” 

This report responds to the four criticisms of Reading Recovery 
highlighted in the Internet letter: effectiveness, cost-effectiveness, 
assessment measures, and change in response to research. 

Effectiveness: Section I reviews research and evaluation on 
Reading Recovery effectiveness for children and provides a 
closer look at the nature of evidence cited by the Internet 
letter. 

Cost-effectiveness: Section II discusses a broader view of 
cost-effectiveness, long-term versus short-term gains, and 
research documenting superior results for one-to-one inter- 
ventions for lowest-performing first graders. This section 
also provides more detail on the evidence cited by the 
Internet letter. 

Assessment Measures: Section III provides background on 
the assessment measures used by Reading Recovery and 
reviews research documenting subsequent success of 
Reading Recovery students on norm-referenced tests. 

Change in response to research: Section IV reports on 
Reading Recovery's built-in mechanisms for change and 
responds to the false accusation that Reading Recovery 
does not teach phonemic awareness and phonics. 
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SECTION I 



READING RECOVERY IS HIGHLY 
SUCCESSFUL WITH THE 
LOWEST-PERFORMING 
FIRST-GRADE STUDENTS. 



Of all charges leveled against Reading Recovery, the question of 
effectiveness is most serious. The Internet letter authors limited 
the kind of evidence examined and excluded research studies that 
did not support their claims. If a research study cited in the 
Internet letter included evidence that balanced criticisms with 
favorable findings for Reading Recovery, the positive findings 
were not reported. In addition, the Internet letter ignored large 
amounts of evaluation data supporting Reading Recovery. 

This section 

A. provides a more complete review of research on Reading 
Recovery’s effectiveness and subsequent literacy gains, 

B. describes Reading Recovery’s internal evaluation system, 
and 

C. analyzes biases in the evidence cited by the Internet 
letter. 

The information presented here represents a broader view of the 
research such as that recently outlined by Michael Pressley in his 
Oscar Causey Research Address to the National Reading 
Conference (2001). 



Substantial evidence 
documents Reading 
Recovery s success with the 
lowest-performing first-grade 
students in a wide range of 
educational settings. 



A. Research demonstrates Reading Recovery 
effectiveness. 

Substantial evidence documents Reading Recovery’s success with 
the lowest-performing first-grade students in a wide range of edu- 
cational settings. This evidence is provided by a variety of study 
designs, including experimental and control group studies both by 
researchers associated with Reading Recovery and by independent 
researchers. 



1. Research in peer-reviewed journals 



The following research published in peer-reviewed journals docu- 
ments Reading Recovery’s effectiveness with first-grade children. 
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Programs with the most 
comprehensive models 
of reading — the most 
complete instructional 
interventions — have greater 
impact than programs 
addressing only a few 
components of the reading 
process, and Reading 
Recovery and Success for All 
include several components. 

(Finding of a 1993 study by 
Wasik and Slavin) 







a. Wasik and Slavin (1993) considered the effectiveness of 
five tutorial programs from two perspectives: empirical and 
pragmatic. The authors reviewed quantitative and qualita- 
tive research on Reading Recovery, Success for All, 
Prevention of Learning Disabilities, Wallach Tutoring 
Program, and Programmed Tutorial Reading. The authors' 
general conclusions across programs were 

• Programs with the most comprehensive models of read- 
ing — the most complete instructional interventions — 
have greater impact than programs addressing only a few 
components of the reading process, and Reading 
Recovery and Success for All include several components. 

• Using tutors is not enough; the content of the program 
and the instructional delivery may be important 
variables. 

• Using certified teachers obtains substantially better 
results than using paraprofessionals. 

The authors' specific conclusions about Reading Recovery 
included 

• Reading Recovery brings the learning of many of the 
lowest-achieving students up to average-achieving peers. 

• Effects of Reading Recovery are impressive at the end of 
the implementation year, and effects are maintained for 
at least two years. 

• Evaluation results on lasting effects are positive but 
complex. 

• Only Reading Recovery has attempted to assess imple- 
mentation and its effect on outcome data. 

Although the authors raised some methodological issues 
about Reading Recovery research and about students 
served, they concluded that the rapidly expanding use of 
Reading Recovery throughout the United States shows that 
the program is practical to use. 

b. Center, Wheldall, Freeman, Outhred, and McNaught 
(1995) evaluated the effectiveness of Reading Recovery 
schools in New South Wales. Low-achieving children were 
randomly assigned to either Reading Recovery (n=31) or a 
control group (n=39) of low-progress students who had 
not entered Reading Recovery by November. A third 
group (n=39) consisted of students from five matched 
schools. By the end of the study, sample sizes were 23, 16, 
and 32 respectively. Measures used were Clay's Diagnostic 
Survey, Burt Word Reading Test, Neale Analysis of 
Reading Ability, Waddington Diagnostic Spelling Test, 
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Phonemic Awareness Test, Cloze Test, Word Attack Skills 
Test, and Woodcock Reading Mastery. 

At post-test evaluation (15 weeks after the pre-test) an 
independent assessment showed that Reading Recovery 
students scored significantly higher on all tests measuring 
reading in context and in isolation. Of the eight measures 
reported, the only ones that did not differ significantly 
were a cloze test and a phonemic awareness measure. At 
short-term maintenance (15 weeks after the post-test) the 
Reading Recovery control group still scored significantly 
higher than the control group on six of the eight measures, 
including Clays text reading measure and several standard- 
ized measures of text and word reading. At this point the 
Reading Recovery group also scored significantly higher 
than the control group on phonemic awareness. 

The study’s published results for medium-term mainte- 
nance (12 months after the post- test) appear to have 
errors. The authors report “no overall significant group 
effect, A(8,30) = 0.262, p = .0268)” (p. 253). There appear 
to be several typos and errors in this statistical statement 
beyond the inclusion of an additional closing parenthesis. 
An F value of 2.62 would match the probability level of 
.0268. Since the authors state that “significant multivariate 
results (alpha = 0.05) were followed up by univariate pair- 
wise multiple comparisons (alpha = 0.01)” (p. 250), the 
conclusion should be that the MANOVA revealed an over- 
all significant group effect in favor of Reading Recovery. 
Still, the only univariate result was for text reading. The 
authors point out that the reduced difference between the 
Reading Recovery and control groups found in the 12- 
month follow-up could be due to the fact that 15 of the 
31 control group students (probably those with the lowest 
scores) had been eliminated from the control group to 
receive Reading Recovery instruction. 

The study provides strong, independent replication of the 
pattern of results found in other research and in the U.S. 
national evaluation data for all participating students. The 
authors state that their “results clearly indicate that low- 
progress students, exposed to 15 weeks of Reading 
Recovery, outperformed control students on Clay book- 
level and Burt Word Reading tests and on all Set 2 tests 
which measure reading and writing words in context and 
isolation” (p. 256). Despite a number of qualifications 
related to metalinguistic measures, the article reports inde- 
pendently measured and extremely large effect size for text 
reading, 3.05 and 1.55 for post-test and short-term main- 
tenance respectively (p. 253). 
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c. Iversen and Tunmer (1993) conducted a study to deter- 
mine whether the Reading Recovery program would be 
more effective if systematic instruction in phonological 
recoding skills were incorporated into the program. Three 
matched groups of 32 at-risk readers were compared: 

• children taught by teachers who received Reading 
Recovery training, 

• children taught by teachers who received Reading 
Recovery training that included phonological recoding 
skills as part of the lesson, and 

• children who received a standard intervention (not 
Reading Recovery), 

Measures included all six tasks of the Diagnostic Survey, 
Dolch Word Recognition Test, Yopp-Singer Phoneme 
Segmentation Test, Phoneme Deletion Test, and 
Pseudoword Decoding Task, 

The critical finding in this study was that the two Reading 
Recovery groups performed at very similar levels when 
Reading Recovery lessons were successfully completed (dis- 
continued), Both groups performed much better on all 
measures than children in the standard intervention group, 
and they often performed significantly better than class- 
room controls (especially on phonological segmentation 
and phoneme deletion). Results revealed that the modified 
Reading Recovery group reached levels of performance 
required for discontinuing more quickly than the standard 
Reading Recovery group. Authors acknowledged that both 
the standard and modified Reading Recovery programs 
included explicit instruction in phonological awareness, 

d. Sylva and Hurry (1996) evaluated the effectiveness of two 
different interventions (Reading Recovery and Phonolog- 
ical Training), Their study included almost 400 children 
from seven English local authorities. Although the sample 
was diverse, inner city children were over-represented. The 
schools included 22 Reading Recovery schools, 23 
Phonological Intervention schools, and 18 control schools. 
The measures used included the British Ability Scale Word 
Reading, Neale Analysis of Reading, Clays Diagnostic 
Survey (five tasks). Assessment of Phonological Awareness, 
British Ability Scale Spelling, and background information 
on each child. 

During the intervention year, the effect of the phonologi- 
cal intervention was more specific than Reading Recovery, 
enhancing childrens phonological awareness and influenc- 
ing their letter identification, dictation, and writing vocab- 
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ulary, but not their text reading skills. Reading Recovery 
children made significantly more progress than the control 
groups on every measure of reading. At the end of the sec- 
ond year, the effects of phonological intervention were still 
evident in enhanced word reading scores, but there was no 
effect on comprehension. In comparison, the Reading 
Recovery children were still six months ahead of the con- 
trol children on word and text reading. 

In their long-term follow-up four years later, Hurry and 
Sylva (1998) concluded that Reading Recovery was still 
effective because almost 70% of the children who had 
received Reading Recovery were still within the average 
band of their class in Grade 6, while only 55% of those 
who received the phonological intervention were within 
the average class band. Reading Recovery was particularly 
effective at helping children who were most socially disad- 
vantaged and who were the weakest readers at age 6. While 
Reading Recovery was more expensive than the phonologi- 
cal intervention in the year of delivery, over the pupils’ 
whole elementary school career it cost only 10% more 
than the general remedial support provided in control 
schools. The authors concluded that time-limited interven- 
tion is not so expensive in the longer term. 

Pinnell, Lyons, DeFord, Bryk, and Seltzers (1994) study 
systematically compared Reading Recovery to three other 
instructional models of early intervention. In this study 
(N=324) the lowest-achieving first-grade students from 40 
different schools in 10 different school districts were ran- 
domly assigned within schools to one of five groups: 

• Reading Recovery, 

• a Reading Recovery-like intervention with partially 
trained teachers, 

• a skills-based individual intervention, 

• small group instruction offered by Reading Recovery 
teachers, or 

• a control group. 

Measures included those used in Reading Recovery as well 
as generally known reading tests (Gates-MacGinitie 
Reading Test and Woodcock Reading Mastery). The study 
employed a formal experimental design that used split 
plots to control effects that may result from differing cul- 
tures among school districts or individual schools. The dif- 
ficulty of small standard errors in analysis of data at the 
student level was addressed by using the Hierarchical 
Linear Model for data analysis. The results of the study 
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Reading Recovery subjects 
performed significantly 
better than any other 
treatment and comparison 
group on all measures. 
Essential differences were 
related to individual 
instruction, the lesson 
framework (combination of 
techniques), and teacher 
training. 

(Findings of 1994 study 
by Pinnell, Lyons, DeFord, 
Bryk, and Seltzer) 
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“Thus, in answer to the 
question 'Does Reading 
Recovery work?,’ we must 
respond in the affirmative. 

It is clear that many 
children leave the program 
with well-developed reading 
strategies, including 
phonemic awareness and 
knowledge of spelling. 
Although some initially 
low-achieving students will 
succeed without Reading 
Recovery, evidence indicates 
that many who would not 
succeed do so as a result of 
this intervention.” 

(Shanahan and Barr, 1995) 



were definitive: Reading Recovery subjects performed sig- 
nificantly better than any other treatment and comparison 
group on all measures. Essential differences were related to 
individual instruction, the lesson framework (combination 
of techniques), and teacher training. 

f. Pinnell (1997) reviewed quantitative and qualitative 
research studies performed by Reading Recovery practi- 
tioners that have demonstrated positive effects of Reading 
Recovery on reading outcomes (including generally recog- 
nized measures), successful replicability in diverse settings, 
positive effects on home relations, improved teacher behav- 
iors and teacher learning, and maintenance of learning 
gains over time. 

g. Shanahan and Barr (1995) published a comprehensive and 
independent evaluation of Reading Recovery. Although the 
Internet letter cites this evaluation as evidence of the inef- 
fectiveness of Reading Recovery, it in fact offers substantial 
support for Reading Recovery’s effectiveness. 

Evidence firmly supports the conclusion that Reading 
Recovery does bring the learning of many children up 
to that of their average-achieving peers. Thus, in ans- 
wer to the question ‘‘Does Reading Recovery work?,” 
we must respond in the affirmative. It is clear that 
many children leave the program with well-developed 
reading strategies, including phonemic awareness and 
knowledge of spelling. Although some initially low- 
achieving students will succeed without Reading 
Recovery, evidence indicates that many who would not 
succeed do so as a result of this intervention, (p. 989) 

That Reading Recovery has been so successful is lauda- 
tory. It has proven to be a robust program, both in 
terms of its consequences for student learning and in 
replicability across sites. Further, it has been a signifi- 
cant force in shaping the way we view early literacy 
development, (p. 992) 

In summary, many research studies published in peer-reviewed 
journals document Reading Recovery’s effectiveness with first- 
grade children. Independent researcher Elfreida Hiebert (1994) 
wrote that “a high percentage of Reading Recovery tutees can oral- 
ly read at least a first grade text at the end of Grade I.... Once a 
program is in place, there appears to be considerable fidelity in the 
results” (p. 21). This is high praise when one considers that 
Reading Recovery children are the lowest literacy achievers at the 
beginning of first grade. 
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2. Research and evaluation studies on improved gains in 
later grades 

Many research and evaluation studies demonstrate that Reading 
Recovery students maintain and improve their gains in later 
grades. The ones listed here have used widely accepted standard- 
ized measures or state assessment measures, or both. 

a. Rowe (1995), an Australian researcher, studied the progress 
made in reading by children from school entry to Grade 6 
in Victoria, Australia. The sample included 5,092 students 
and 256 classes in 92 schools. The researchers intent was 
not specifically to study Reading Recovery, but informa- 
tion on Reading Recovery’s effectiveness emerged as an 
outcome. The longitudinal design involved repeated meas- 
ures nested within classes and schools and repeated meas- 
ures on schools. The second design involved cross sections 
of students nested within schools that were changing over 
time. Rowe used several measures to gather student infor- 
mation: Reading Achievement, Primary Reading Survey 
Test, Test of Reading Comprehension, English Profile, and 
Reading Bands. 

Rowe found that Reading Recovery children benefited 
notably from participation in the intervention. Reading 
Recovery appeared to be meeting its intended purpose for 
those children involved. By Grades 5 and 6, Reading 
Recovery students were distributed across the same score 
range as the general school population, but with fewer low 
scores. Rowes analysis provided evidence that Reading 
Recovery had removed the tail-end of the achievement dis- 
tribution. 

b. Brown, Denton, Kelly, and Neal (1999) used two stan- 
dardized tests to assess California students’ continuing 
achievement through fifth grade. Researchers measured 
achievement of 760 students who were served in Reading 
Recovery between 1993 and 1998. Student performance in 
second through fifth grades was assessed using the Iowa 
Tests of Basic Skills and Stanford Achievement Test, Ninth 
Edition. The authors reported that ‘more than three- 
fourths of the children who successfully completed 
Reading Recovery achieved standardized test scores in the 
average or above average range” (p. 10). Considering that 
these Reading Recovery students began at the bottom of 
their class in first grade, their subsequent progress through 
fifth grade was impressive. 

c. Askew, Kaye, Frasier, Mobasher, Anderson, and 
Rodriguez (2002) collected longitudinal data on former 
Reading Recovery children in 45 randomly selected 
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“A high percentage of 
Reading Recovery tutees 
can orally read at least a 
first grade text at the end of 

Grade 1 Once a program 

is in place, there appears to 
be considerable fidelity in 
the results.” 

(Heibert, 1994) 



page 21 



schools through fourth grade. The study focused on dis- 
continued children (those students who met the rigorous 
criteria for success) in order to see if children who reached 
average performance in Grade 1 continued to score within 
average ranges in subsequent years. At the end of fourth 
grade, a large majority of these children had scores consid- 
ered to be average or meeting passing criteria on standard- 
ized (Gates-MacGinitie Reading Test) and state assessment 
measures, a very satisfactory outcome in their school set- 
tings. They were generally perceived by their fourth-grade 
teachers as performing within average ranges of their class- 
rooms. Relatively few were placed in tertiary or remedial 
settings. Findings match Juels (1988) conclusions that 
children who are average readers in Grade 1 remain aver- 
age readers in Grade 4, supporting the need for interven- 
tion in first grade. 

d. Schmitt and Gregory (2001) conducted a statewide 
follow-up study in Indiana to examine subsequent per- 
formance of former successful Reading Recovery children 
in Grades 2, 3, and 4. The study demonstrated that the 
majority of the 271 randomly selected Reading Recovery 
children were performing as well as a random sample of 
277 of their grade-level peers on the vocabulary and com- 
prehension subtests of the Gates-MacGinitie Reading Test. 
Similarly, third-grade student results on the Comprehen- 
sive Test of Basic Skills-5/Terra Nova Form B reflected a 
normal distribution with a mean at the 45th percentile 
for these students who had been the lowest-achieving 
first graders. 

e. Pinnell (1989) evaluated two cohorts of students. The pur- 
poses of the study were to explore whether Reading 
Recovery could succeed with low-achieving children and to 
determine whether those children maintained their gains. 
The lowest-achieving children were randomly assigned 
either to Reading Recovery or to a control group served 
daily in individual lessons taught by a trained paraprofes- 
sional (not a Reading Recovery teacher). Both groups were 
compared with a random sample of average and high- 
progress first graders as an indication of average progress. 
The study used all six tasks of Clays Diagnostic Survey,^ a 
writing sample, and the Comprehensive Test of Basic Skills 
(two subtests). Pinnell found that in the full Reading 
Recovery program, Reading Recovery children scored sig- 
nificantly better than control children on seven of the nine 



2 Until the Observation Survey was published in 1993, it was known as the Diagnostic 
Survey (Clay, 1985). 
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diagnostic measures at the end of first grade. They com- 
pared well with the random sample group. Reading 
Recovery children were followed in second and third grade 
to determine their performance in text reading in subse- 
quent years. Reading Recovery children remained superior 
in comparison with the control group. 

In addition to these studies, many school districts and teacher 
training sites conduct their own independent analyses through 
their research departments. Typically these analyses use the stan- 
dardized measures that the districts use with all children. One such 
study (Quay, Steele, Johnson, & Hortman, 2001) in a Georgia 
school district compared Reading Recovery children with a control 
group who were equivalent in gender, ethnicity, and achievement. 
“At the end of the school year, multivariate and univariate analyses 
of variance indicated that the Reading Recovery children were sig- 
nificantly superior to the control group children on: (a) The Iowa 
Test of Basic Skills Language Tests; The Gates MacGinitie Reading 
Test; (c) the six tests of An Observation Survey of Early Literacy 
Achievement; (d) classroom teachers’ assessments of achievement 
in mathematics, oral communication, reading comprehension, and 
written expression; (e) classroom teachers’ ratings of personal and 
social growth in work habits, following directions, self-confidence, 
social interaction with adults, and social interaction with peers; 
and (f) promotion rates” (p. 7). These results are especially signifi- 
cant considering that all teachers in this study were in their initial 
training year. 

3. Studies on improved student self-esteem 

Two studies published in refereed journals have revealed that 
Reading Recovery students experience gains in self-concept. 

a. Cohen, McDonnell, and Osborn (1989) studied the 
impact of Reading Recovery on students’ beliefs regarding 
their competence and capacity to direct their own learning 
activities. They used causal attribution (Weiner, 1972) and 
self-efficacy (Bandura, 1977) to support the theoretical 
framework. Participants included 138 first graders divided 
among the following groups: 50 were in Reading Recovery, 

48 were in remedial reading groups of five or six students 
each, and 40 were randomly selected from their higher- 
achieving classmates. 

After the interventions, children were tested on two scales 
to measure attributions and self-efficacy. Results demon- 
strated that successful Reading Recovery children had pro- 
files similar to high-achieving students, and they more 
readily attributed their success in school to ability, effort, 
and mood than did the students in the remedial groups. 
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‘‘School districts that choose 
to implement and maintain 
a Reading Recovery 
program would reap 

considerable benefits 

Not only will the Reading 
Recovery participants most 
likely become independent 
readers, they will also most 
likely become more 
confident, positive, 
self-accepting, proud, 
adaptable, and eager to 
complete tasks.” 

(Rumbaugh and Brown, 2000) 



The Reading Recovery students also judged themselves to 
be more competent on school-related tasks (self-efficacy) 
than the other lov^- achieving students. These results sup- 
port the notion that children have positive self-esteem 
when they leave Reading Recovery. 

b. Rumbaugh and Brown (2000) studied the effects of 
Reading Recovery participation on students’ self-concept. 

The treatment group was comprised of 57 students from 
nine elementary schools who were selected for Reading 
Recovery instruction in the first week of school. The 46 
students in the control group had diverse reading and writ- 
ing abilities and were not enrolled in any reading interven- 
tion or in special education. The control group came from 
a single elementary school. 

All participants were administered the Joseph Pre-School 
and Primary Self-Concept Screening Test in early Septem- 
ber prior to the treatment and again in mid-December. 

There were statistically significant differences between 
Reading Recovery students and control students on the 
Global Self-Concept and Significance domain scores. 

Hence, the authors concluded 

• Reading Recovery participation does affect positively stu- 
dents’ Global Self-Concept scores. 

• The meaningful effect of Reading Recovery participation 
on students’ self-concept is related to the additional 
attention, or Significance domain, that students receive 
during several months of Reading Recovery. 

• The initial positive effect on students’ self-concept can- 
not be attributed to increased growth in independence or 
cognitive factors. 

Based on their results, Rumbaugh and Brown concluded: 

School districts that choose to implement and maintain 
a Reading Recovery program would reap considerable 
benefits. One of the systemic advantages could be that 
the districts gain students who experience improved 
self-concepts due to enhanced feelings of significance. 

Not only will the Reading Recovery participants most 
likely become independent readers, they will also most 
likely become more confident, positive, self-accepting, 
proud, adaptable, and eager to complete tasks, (p. 28) 

As in any program or in any classroom, children who are failing 
are likely to have self-esteem difficulties. Yet one of the first reports 
from both parents and first-grade classroom teachers is about the 
change in Reading Recovery students’ self-esteem when they are 
making progress in Reading Recovery. 

Q What Evidence Says About Reading Recovery 
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B. Evaluation evidence supports Reading Recovery 
effectiveness. 

Reading Recovery is very successful with its targeted population, 
first-grade students having most difficulty with early literacy 
acquisition — the lowest achievers in the class. This success has 
been carefully documented for 17 years in data gathered and ana- 
lyzed on every single U.S. student enrolled in Reading Recovery. 

All data are reported to the National Data Evaluation Center 
located at The Ohio State University. 

Findings are reported in national, state, and district-level technical 
reports that are designed to present annual evaluations of system- 
atic, simultaneous replications of Reading Recovery. These reports 
are widely disseminated to state legislators, state boards of educa- 
tion, local school boards, superintendents, and principals. 

Since 1984, Reading Recovery in the United States has collected 
and analyzed data and reported results for more than one million 
children. Results confirm Reading Recovery’s success with the 

lowest-performing first graders, as detailed in the table below. ^ 

^ ^ Reading 

Recovery in the United 
States has collected and 
analyzed data and reported 
results for more than 
one million children. 



Facts About Reading Recovery Student Success in 2000-2001* 

149,009 students served 

112,814 received a full series of lessons 

86,009 discontinued lessons (successful completion) 

36,195 did not receive a full series of lessons 

Percentage of Students Who Successfully Completed Lessons 

78% of students who received a full series of lessons 
successfully met discontinued criteria 
59% of students served, even for one lesson, 
successfully met discontinued criteria 

Notes: 

Full series of lessons: Those who received at least 20 weeks of the 
30-minute daily lessons or successfully completed lessons before 
20 weeks. 

Did not receive a full series of lessons: Most often those students 
who do not begin lessons until late in the year. 

Discontinued: Those who successfully meet the rigorous criteria 
to be discontinued (released) from Reading Recovery during the 
school year or at the time of year-end testing. 

* Does not include Descubriendo la Lectura students. Results for the 3,232 
Descubriendo la Lectura students served in 2000-200 1 are comparable to 
Reading Recovery. 
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Every child served in 
Reading Recovery even if for 
only one day is counted and 
reported in data from the 
National Data Evaluation 
Center. 



These percentages translate into powerful realities. For example, of 
the 86,009 students scoring at or above grade-level expectations 
on post-treatment measures of literacy skills, only 113 (0.13%) 
were placed in special education settings for reading support, and 
only 23 were assigned to special education settings for writing 
instruction at the end of first grade. At the same time, only 194 
(0.26%) were retained in Grade 1 due to reading performance. 

In fact, placement in special education or retention in Grade 1 for 
reading difficulties was found to occur more frequently in the 
randomly selected, non-treatment group representing average 
performance than in the group of children successfully served by 
Reading Recovery. 

The children who are not discontinued and are recommended for 
further evaluation also make progress in Reading Recovery. A 
review of scores on measures that exhibit a ceiling effect is inform- 
ative. Measures with a ceiling effect can be treated as criterion 
measures, that is, performance levels that all first-grade children 
should reach some time during the first grade. For example, dur- 
ing fall testing for the phonemic awareness measure (Hearing and 
Recording Sounds in Words), recommended students accurately 
recorded only 6.6 sounds on average compared to 23.9 on average 
for the random sample children. In the spring testing, the recom- 
mended students recorded 31.8 sounds and the random sample 
35.1 sounds. Although this group of recommended students did 
not meet Reading Recovery’s rigorous criteria for being discontin- 
ued, the evidence suggests that these children met the criterion 
level on several measures (NDEC, 2002). 

1. Reading Recovery counts every child. 

Authors of the Internet letter claim that “studies conducted by 
researchers associated with Reading Recovery typically exclude 
25-40% of the poorest performing students from the data analy- 
sis” (paragraph 3). Two possible origins of this argument are 
hypothesized. First, a 1995 article (Center et al.) asserted that 
Clays studies had excluded about 30% of children who were 
either removed or not discontinued from the program. However, 
Clays 1979 data clearly negate this claim: No children were 
dropped from her analyses. Clay responded to this claim in a pub- 
lished letter in Reading Research Quarterly (1997). Yet the Center 
et al. accusation has been carried forward on an ongoing and inac- 
curate basis by other researchers. 

A second possibility is that the Internet letter authors were refer- 
ring to studies which have included only discontinued children, 
those who have successfully.completed Reading Recovery lessons. 
For some research, it is very appropriate to study specific groups of 
Reading Recovery children to answer identified research questions. 
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For example, some researchers have studied children who success- 
fully completed lessons in order to determine if children who 
reach average performance at the end of Grade 1 maintain that 
average status in subsequent years. This is a very legitimate 
research question. To answer this question, no researcher would 
include every child; some children would have received few lessons 
and comparisons would be inappropriate. 

Regardless of the confusion leading to the claim that Reading 
Recovery excludes poorest-performing students from data analysis, 
it is important to acknowledge that every child served in Reading 
Recovery, even if for only one day, is counted and reported in data 
from the National Data Evaluation Center. All evaluation data are 
inclusive of all children, regardless of outcome status. The broad 
accusation made in the Internet letter is misleading at best. 

2. Reading Recovery serves first-grade children with the 
greatest literacy need. 

The Internet letter confuses the description of the target popula- 
tion that Reading Recovery serves. No prerequisite skills are 
required for being served by Reading Recovery. Reading Recovery 
is a safety net intervention for first graders who are having difficul- 
ty with early reading and writing. It is an important component of 
a schools comprehensive literacy program. Reading Recovery pro- 
vides instruction for the lowest-achieving children first. Annual 
Reading Recovery data clearly demonstrate that those served first 
in the school year have the lowest scores on literacy measures at 
the beginning of the first-grade year. 

All students with the lowest scores for their school enter the 
program and represent the target population (Clay & Tuck, 

1991). Outcomes cannot be predicted reliably by any measure at 
entry. Any attempt to restrict access to Reading Recovery service 
based on low entry test scores would deny service to children 
most in need and many who later successfully complete Reading 
Recovery lessons. 

Reading Recovery has two positive outcomes. First, Reading 
Recovery students successfully complete lessons (discontinue) 
when they are judged to have met the criteria to participate in 
their classroom literacy instruction at an average level and to effec- 
tively apply strategies that will support future literacy learning. 

Second, those students who have not achieved these rigorous crite- 
ria after 20 weeks of lessons have still received an intensive period 
of diagnostic teaching. Recommendation for further evaluation is 
a positive step because evaluators have much more information 
about student strengths and weaknesses than was available before 
Reading Recovery. In addition, students have learned many new 
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skills and strategies even though they did not meet criteria for suc- 
cessful completion of lessons. 

Many Reading Recovery studies disaggregate the data by these cat- 
egories (discontinued and not discontinued) to provide an indica- 
tion of the proportion of children returned to average levels of 
performances versus those identified as needing additional evalua- 
tion or alternative support. 

3. Reading Recovery methodology follows rigorous stan- 
dards for evaluation research. 

The criticism that Reading Recovery does not follow intent to 
treat methodology is misplaced (paragraph 3). The intent to treat 
methodology is used in experiments or trials where research sub- 
jects are randomly assigned to one of several treatments. In con- 
trast, Reading Recovery serves children identified by the school as 
the lowest-achieving first-grade students and reports on the total 
population served in Reading Recovery. Reading Recovery lessons 
are not clinical trials with random assignment but rather services 
delivered to an entire population. 

An important aspect of intent to treat, however, is to account for 
every study subject, and Reading Recovery does meet that criteri- 
on. The only students excluded from National Data Evaluation 
Center reports are those for whom outcome status is missing, and 
that fact is clearly reported. In the 2000-2001 year, the annual 
data accounted for 99.93% of children served. Outcome data were 
missing for only 90 of the 152,241 children served by Reading 
Recovery and Descubriendo la Lectura (NDEC, 2002). 

Data are reported for every child enrolled in Reading Recovery. 
Students are tested pre- and post-treatment (before and after being 
served) as well as at year-end. At the end of each child’s series of 
Reading Recovery lessons, a status category is assigned. The five 
status categories are 

• Discontinued: The child has successfully completed the 
program and is able to benefit from classroom instruc- 
tion. 

• Recommended action after a full program of 20 weeks: 

The child was recommended for further evaluation and 
consideration of other support services. 

• Incomplete program at year-end: The school year ended 
before the child had time to complete the program. 

• Moved while being served. 

• None of the above: This category is used only in special 
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circumstances when the child has to be removed from 
the program for very unusual reasons (such as return to 
kindergarten). 

In 2000-2001, of all 149,009 Reading Recovery students (exclud- 
ing students in Descubriendo la Lectura) served, regardless of the 
number of lessons received, 59% discontinued their lessons suc- 
cessfully, 17% were recommended for further evaluation, 16% had 
incomplete lessons at the end of the school year, 5% moved before 
lessons could be completed, and 3% were classified as none of the 
above. Of the children with the full series of Reading Recovery les- 
sons, 78% successfully discontinued their programs. Results for 
the 3,232 children served in Descubriendo la Lectura were compa- 
rable to Reading Recovery results. 



C. Bias in the Internet letter is analyzed. 

Given all the evidence for the effectiveness of Reading Recovery, 
how do the authors of the Internet letter conclude that the pro- 
gram doesn’t work for the lowest-performing students? First, they 
limit their view of acceptable evidence to experimental, random 
control group experiments. This allows them to ignore the exten- 
sive data collected annually and publicly reported on the progress 
of every single student enrolled in the program. To advance their 
political agenda, the authors of the Internet letter selectively report 
and distort the limited set of experimental studies they present to 
their colleagues, politicians, and the public for their support. 

The clearest instance of this distortion is in their use of the 
Elbaum et al. (2000) meta-analysis. They state, “In fact, for the 
poorest readers, empirical synthesis of 'in-house’ and independent 
studies indicates that Reading Recovery is not effective. In Elbaum 
et al. (2000), the gains for the poorest readers instructed by 
Reading Recovery were almost zero” (paragraph 3). Why is this a 
distortion? First, it ignores the major finding of this meta-analysis 
that the effect of Reading Recovery on student performance was 
large and significant. 



Second, it confuses a post-treatment outcome variable, discontin- 
ued versus not discontinued, with a pre-treatment aptitude vari- 
able. Students are discontinued from Reading Recovery service 
when they are judged to have met the criteria to participate in 
their classroom literacy instruction at an average level and have a 
set of strategies judged to be sufficient to support future literacy 
learning. Students who are not discontinued have not achieved 
these criteria with 20 weeks of lessons and are judged not likely to 
meet these criteria with an additional week or two of lessons. 
Many Reading Recovery studies disaggregate the data by these cat- 
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egories to provide an indication of the proportion of children 
returned to average levels of performances versus those identified 
as needing long-term literacy support. Both of these outcomes 
have positive implications for the efficient use of future education- 
al funds. 



Authors of the Internet 
letter limit their view of 
acceptable evidence to 
experimental, random 
control group experiments. 
Then they selectively report 
and distort the limited set 
of experimental studies 
they present. 



The authors of the Internet letter imply that these not-discontin- 
ued students were those who entered the program with the lowest 
scores. This is not the case. All students enter the program with 
low scores and represent the target population. Again, the Elbaum 
et al. (2000) study indicates that for the total group the effects of 
Reading Recovery are large and significant. The outcome category 
cannot be reliably predicted by any measure at entry. Any attempt 
to restrict access to Reading Recovery service based on entry tests 
would deny service to those children most in need of service, 
many who show substantial benefit from that service. 

Elbaum et al. (2000) state, “For Reading Recovery interventions, 
effects for students identified as discontinued were substantial, 
whereas effects for students identified as not discontinued were 
not significantly different from zero” (p. 605). The Internet 
letter emphasizes the small number of students who did not 
make progress while it ignores the fact that the majority of stu- 
dents made substantial progress. It also implies that the not- 
discontinued students were the ones with the lowest entry scores. 
This is not true. The interpretation of this information in the 
Internet letter seems biased. 



D. Summary 

Studies and research reviews in peer-reviewed journals document 
the effectiveness of Reading Recovery for the lowest-performing 
first graders. In addition, evidence of subsequent gains is substan- 
tial. Moreover, self-esteem studies reveal that Reading Recovery 
children improve self-efficacy scores as a result of the treatment. 

Authors of the Internet letter limit their view of acceptable evi- 
dence to experimental, random control group experiments. This 
allows them to ignore the extensive data collected annually and 
publicly reported on the progress of every single student enrolled 
in the program. Then they selectively report and distort the limit- 
ed set of experimental studies they present. 

The National Data Evaluation Center collects, analyzes, and 
reports data on every child served each year. Results are impressive 
for these children who initially have the most difficulty with litera- 
cy learning. What other program provides such external and inter- 
nal evidence of success with the lowest-achieving first graders? 
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SECTION II 



COST-EFFECTIVENESS 
IS A COMPLEX CONCEPT 
IN EDUCATION. 



Given the competing demands on education budgets at federal, 
state, and local levels, it is important to address literacy learning in 
cost-effective ways. To date, however, no cost-effectiveness analysis 
comparing alternative early literacy interventions has been con- 
ducted (for example, small group instruction compared with one- 
to-one instruction). This is not surprising since cost-effectiveness 
analysis is not a widely used evaluation tool in education decision 
making, although it is often discussed (Hummel-Rossi &C 
Ashdown, 2002; Levin & McEwan, 2001). 

The Internet letter states that “Reading Recovery is not cost- 
effective because the developers require one-to-one interventions 
by highly trained teachers” (paragraph 4). This assertion could be 
true only if working with low-achieving children in groups was as 
effective as working with them one-to-one. 

This section 

A. reviews cost-effectiveness issues and the long-term bene- 
fits of Reading Recovery, 

B. documents research demonstrating the efficacy of one- 
to-one interventions for the lowest-performing first- 
grade students, and 

C. analyzes the quality of research that the Internet letter 
cites to support claims for superior results of group 
interventions. 



A. Cost-efifectiveness cannot be oversimplified. 

1. Long-term versus short-term benefits and the population 
served 

To describe any program as expensive or not cost-effective is 
misleading without providing crucial information such as which 
students the programs target and what results are sought relative 
to the performance of other students. Another important 
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consideration in cost-effectiveness is the longer-term benefit of 
the intervention. 



In the case of Reading 
Recovery, the long-term 
benefits of literacy 
achievement may 
significantly outweigh the 
short-term cost of instruction 
and teacher preparation. 
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In the case of Reading Recovery, the long-term benefits of literacy 
achievement may significantly outweigh the short-term cost of 
instruction and teacher preparation. Reading failure is a predictor 
of academic underachievement. As the achievement gap widens, 
the potential for future economic, social, and psychological prob- 
lems increases for this group of children (Pikulski, 1994). The 
significant number of children — estimated to be 20% — ^who expe- 
rience early reading failure are at risk of not reaching acceptable 
standards throughout their academic careers (Hill & Crevola, 
1997). The price of achieving literacy for all students should be 
weighed against the social ills associated with literacy failure and 
drop-out rates. 

The role of prevention (Pianta, 1990) is essential when consider- 
ing long-term costs associated with literacy failure. Reading 
Recovery is a secondary prevention that targets an identified group 
of the first-grade population (lowest-performing) with the highest 
likelihood of experiencing literacy failure, even within good pri- 
mary prevention efforts (classroom instruction). By targeting the 
lowest-achieving first graders, two positive outcomes are possible: a 
child achieves successful literacy performance or is identified for 
further assessment and possibly for an appropriate alternative 
instructional program known as tertiary prevention. 

By intervening early, Reading Recovery reduces referrals and 
placements in special education (NDEC, 2002), limits retention, 
and has demonstrated lasting effects. Retention and special educa- 
tion referral also have substantial price tags. The local costs of pro- 
viding Reading Recovery services for 12 to 20 weeks will be sub- 
stantially less than these costs, particularly when the majority of 
Reading Recovery children sustain their literacy learning gains. 

The Indiana Education Policy Center at Indiana University con- 
ducted an independent study of Reading Recovery in the state and 
reported that the programs impact resulted in reduced grade- 
retention rates in all schools and larger gains in passing rates on 
the Comprehensive Test of Basic Skills-5/Terra Nova Form B two 
years after the intervention in high-poverty schools (Manset, St. 
John, & Simmons, 2000). 

Other outcomes associated with literacy achievement, although 
not so easily measured or valued, must be considered when 
evaluating costs. Barnett (1993), in an economic evaluation of 
human service interventions, called these variables “qualitative 
residual outcomes” (p. 95). Within the context of Reading 
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Recovery, a qualitative residual outcome is the investment in 
teaching skill: teachers can apply their additional skill level in 
instructional contexts other than Reading Recovery. 



Cunningham and Allington (1994) address the complexity of cost- 
effectiveness analysis. 

The criticism most often made of Reading Recovery is that 
it is too expensive and that it requires too much teacher 
training. However, getting these results with the hardest- 
to-teach children leads us to conclude that the teacher 
training is providing the teacher with extraordinary insight 
and skills. It does cost money to hire and train Reading 
Recovery teachers but it also costs money to employ 
transitional-grade teachers (e.g., pre-first classes), resource 
room teachers, and remedial teachers, too. It costs money 

to retain children When you compare the success rate of 

Reading Recovery with other programs that keep children 
for years and never get them reading on grade level, 

Reading Recovery is a bargain! (p. 255) 

2. The difficulty of assessing prevention costs 

Costs in any prevention program are difficult to assess because so 
many factors must be considered: 

• regional cost variables affecting salaries, overhead, and 
more 

• level of need for the service and level of coverage 
provided 

• quality of training for teacher leaders and teachers 

• efficiency and effectiveness factors in program delivery 

• acceptance of program as an integral part of the system 

• sustained gains for children resulting in reducing referrals 
to special education and lower rates of retention at the 
end of first grade 



“When you compare the 
success rate of Reading 
Recovery with other 
programs that keep children 
for years and never get them 
reading on grade level, 
Reading Recovery is a 
bargain!” 

( Cunningham and Allingtony 

1994) 



Most school districts expect that there will be personnel costs and 
costs for training and materials for every program implemented in 
the system, particularly programs targeted to the lowest achievers. 
As Levin and McEwan (2001) point out, decision makers have to 
realize that some children will cost more to educate. Many dis- 
tricts consider in-service training of teachers to be an ongoing 
responsibility of a school district and do not consider Reading 
Recovery training an additional expense. 
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3. Prevention costs in Reading Recovery 

The investment in Reading Recovery reduces the number of chil- 
dren who need ongoing, expensive services. Two school districts 
have calculated the relative costs of Reading Recovery versus the 
costs of first-grade retention, Title I remedial instruction, and spe- 
cial education for children classified as learning disabled. These 
analyses have used district teacher salary figures to calculate both 
the annual and the cumulative amounts of time that a single child 
would be likely to spend in each of the programs. 

a. Lyons and Beaver (1995) conducted a cost comparison 
analysis for first-grade retention in Lancaster, Ohio four 
years after Reading Recovery was implemented system- 
wide. The study revealed that the first-grade retention rate 
dropped from 4.3% (76 of 1,772 students) in the three 
years prior to implementation of Reading Recovery to 
2.9% (63 of 2,123 students) four years after systemwide 
implementation. Using teachers’ salaries and students’ time 
in the program, these figures represented a cost savings of 
$163,020. In addition, the Lancaster district looked at spe- 
cial education placements. In the three years prior to full 
implementation of Reading Recovery, 32 students were 
placed in learning disabilities classrooms at the end of 
Grade 1 or during the first few months of Grade 2. In the 
three years after Reading Recovery implementation, 10 
children were classified as learning disabled. The cost of 
educating one learning disabled student at the time was 
conservatively estimated at $9,100 across four years of 
service compared with the per pupil cost of $1,708 for 
Reading Recovery. The authors found that considerable 
savings were realized after the district established Reading 
Recovery as a prevention program. 

b. Assad and Condon (1996) conducted another cost- 
effectiveness study of Reading Recovery in Fall River, 
Massachusetts. During a 2-year period (1993-1994 and 
1994—1995), the Fall River Reading Recovery program 
served 186 students at an annual per pupil cost of $2,363. 
Added to this was the cost of additional interventions for 
several referred or retained children, for a total implemen- 
tation cost of $483,271. Using the data collected on reten- 
tion, special education, and Title I placement in years prior 
to Reading Recovery implementation, district administra- 
tors estimated that without Reading Recovery, 50% of the 
Reading Recovery students would have been referred to 
special education and 50% would have been referred for 
Title I services. Administrators also estimated that approxi- 
mately 5.7% would have been retained. Using these fig- 
ures, district administrators estimated a 5-year cost of 
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$1,746,145 if Reading Recovery had not been implement- 
ed in the district, for a net savings of $1,262,874. This 
dollar amount, however, does not translate directly into a 
reduction of school department spending. It is an estimate 
of the resources that because of Reading Recovery will not 
be needed for teaching basic literacy skills, thus allowing 
funds to be shifted to meet other important needs. 



B. One-to-one interventions are essential for acceler- 
ated learning of the lowest-performing first graders. 

1 . Evidence supports one-to-one intervention for the 
lowest-achieving first graders. 

One-to-one intervention is more reliable than group programs. 
Solid scientific evidence supports the effectiveness of Reading 
Recovery’s one-to-one tutoring model versus small group instruc- 
tion for the lowest-performing first graders (Dorn & Allen, 1995; 
Pinnell et al., 1994). The individualized tutoring enables the high- 
ly trained Reading Recovery teacher to tailor each lesson to the 
unique needs of each struggling student. This individual tutoring, 
in contrast to rigidly scripted programs for all children, is efficient 
because the teacher does not waste time on what the child already 
knows. The Reading Recovery framework is qualitatively different 
for every child because the teacher makes decisions based on indi- 
vidual needs within each lesson component. The teacher is always 
pushing the boundaries of the learning of the particular child. 

This explicit and intensive instruction would be weakened if 
teacher time was divided among several other children. 

Evidence that small group instruction is just as effective with this 
challenging group of young learners is seriously suspect. For exam- 
ple, the Elbaum meta-analysis cites only two studies that com- 
pared a one-to-one intervention with a small group intervention 
(Acalin, 1995; Evans, 1996). Neither study justified the claim 
made for small group instruction in the Internet letter (see com- 
ments on both studies on pages 39 and 40 of this report). 

The assertion of superior cost-effectiveness of small group instruc- 
tion goes against years of research documenting failure of tradi- 
tional small group remedial instruction and ability grouping to 
close the gap for children from poor and minority backgrounds 
(Allington, 2001; Allington & Cunningham, 2002; Shepard, 
1991). One-to-one tutoring of primary-grade students has been 
demonstrated to be a successful strategy (Wasik & Slavin, 1993). 

Arguments related to one-to-one instruction versus group instruc- 
tion must acknowledge the difficulties of comparing effects of 
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these two delivery systems. For example, many factors could influ- 
ence the outcomes in both one-to-one and small group settings, 
including age of subjects, initial performance of subjects, measures 
used, criteria for success, size and design of the study, duration of 
the intervention, curriculum content, quality of training and peda- 
gogy, length of intervention, subsequent performance, and more. 

Hurry’s (2000) review of intervention research states, ‘‘The 
available evidence on the effectiveness of one-to-one tuition 
[teaching] is very positive, but the curriculum content and peda- 
gogy are also important” (p. 20). Pinnell and her colleagues (1994) 
make a similar argument: One-to-one setting, a lesson framework 
with intensive literacy experiences, and long-term teacher training 
are all necessary but not sufficient to explain the success of 
Reading Recovery. 

We know that not all one-to-one interventions and not all small 
group programs yield positive outcomes for children; therefore, 
consideration of complex factors contributing to success in both 
delivery systems merits attention. It is not simply a question of 
one-to-one versus small group instruction. 

2. The MacArthur Foundation-funded study (Pinnell et al., 
1994) supports one-to-one intervention. 

This well-designed, large scale experimental field study (40 
schools) was designed in response to challenges about the delivery 
system of Reading Recovery. Reading Recovery was systematically 
compared with (1) another one-to-one intervention, (2) a one-to- 
one intervention with teachers who had limited training in 
Reading Recovery, and (3) group instruction based on Reading 
Recovery principles with trained Reading Recovery teachers. 

The lowest children (N=324) in the 40 schools were randomly 
assigned within schools to either one of the four treatments or a 
control group. Researchers at the University of Chicago independ- 
ently analyzed the data. In addition, a renowned national panel of 
researchers not involved in Reading Recovery provided oversight 
for analyzing the results. 

Results were definitive. Reading Recovery subjects performed 
significantly better than any other treatment and comparison 
groups on all measures. Essential differences were related to indi- 
vidual instruction, the lesson framework (combination of tech- 
niques), and teacher training. (See pages 19-20 for further 
description of results.) 
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3. An Arkansas study (Dorn & Alien, 1995) supports one- 
to-one intervention. 

Dorn and Allen reported the simultaneous implementation of 
Reading Recovery and a specially designed small group model. 
Extensive staff development was provided to Reading Recovery 
teachers who taught Reading Recovery (30-minute sessions) for 
part of the day and small groups (45-minute sessions) for part of 
the day. 

The lowest children were served first in Reading Recovery; others 
were placed in groups of five. When a child exited Reading 
Recovery, the lowest child in a small group or the lowest child 
from a classroom was placed in Reading Recovery. Priority was 
given to offering the individual tutoring for children who needed 
it most. 

Data for 231 children were analyzed: 95 received Reading 
Recovery tutoring only, 93 received group services only, and 43 
received a combination of group service and Reading Recovery. Of 
the children receiving Reading Recovery only, 76% were success- 
fully discontinued from service. These were initially the lowest 
children in the study. Of the children receiving only group service, 
30% reached successful levels of reading achievement. These chil- 
dren were initially higher than the Reading Recovery group in 
reading performance. Dorn and Allen concluded that Reading 
Recovery was the most effective program for the lowest children 
who must have individually tailored lessons. 



C. Research cited by the Internet letter is flawed. 

To support its recommendation for change in Reading Recovery 
group size, the Internet letter cites a meta-analysis (Elbaum et al., 
2000) and two unpublished doctoral dissertations (Evans, 1996; 
Iversen, 1997). Although the previous section reviews some aspects 
of the Elbaum meta-analysis, this section focuses on problematic 
findings related to group size, group composition, and fidelity of 
treatments included in the meta-analysis. 

1. Elbaum meta-analysis 

Elbaum, Vaughn, and Moody (2000) attempted a synthesis of 31 
different studies. The goal was to explore relative effects of various 
features of intervention programs. These features included, among 
others, small group versus individual instruction and Reading 
Recovery versus other types of interventions. Of the 31 studies 



Of the children receiving 
Reading Recovery only, 

76% were successfully 
discontinued from service. 
These were initially the 
lowest children in the study. 
Of the children receiving 
only group service, 30% 
reached successful levels of 
reading achievement. 

(Results of 1995 study 
by Dorn and Allen) 
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included in the analysis, 10 purported to include Reading 
Recovery, but at least two of them (McCarthy, Newby, & Recht, 
1995; Iversen & Tunmer, 1993) did not deliver services meeting 
national Reading Recovery standards. 

A significant flaw exists in the Elbaum et al. meta-analysis design. 
In order for meta-analysis results to be fair and effective, studies 
must include substantially similar constructs (Wortman, 1992). 
According to the writers, 31 studies included ‘adult-delivered one- 
to-one instructional interventions in reading for elementary school 
children identified as being at risk for reading failure” (p. 606). 
Fourteen of these studies examined tutoring of children beyond 
first grade. Reading Recovery includes only first-grade children. 
Within the 29 treatment-control studies in the meta-analysis data- 
base, Elbaum and colleagues identified 14 different constructs 
with a total of 54 different variations. For example, 

• At-risk was variously defined as lowest 20th— 30th 
percentile or learning disabled. 

• Instruction was delivered by teachers, college students, or 
volunteers. 

• Five different foci of instruction were defined across the 
various studies. 

• Multiple and complex outcome measures were used in 
the various studies. 

• One-to-one instruction was variously compared to class- 
room instruction and other one-to-one comparison 
groups. 

• Except for the Reading Recovery feature, the authors did 
not separate the studies of first-grade interventions from 
those that looked at older children. 

Another problem area in the Elbaum et al. meta-analysis is the 
treatment of the study by Pinnell et al. (1994). This large-scale 
field study compared three one-to-one treatments (Reading 
Recovery, Reading Recovery-like tutoring with a partially trained 
teacher, and a skills tutoring model) to one group treatment (a 
small group taught by a Reading Recovery teacher). Elbaum inap> 
propriately averaged the effects of the three experimental one-to- 
one treatments. These three one-to-one treatments were very dif- 
ferent approaches and had different outcomes; therefore, averaging 
was inappropriate. 

As evidence that Reading Recovery as a one-to-one tutorial had no 
advantage over small groups, Elbaum et al. cite an unpublished 
doctoral dissertation by Evans (1996) and an unpublished masters 
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thesis by Acalin (1995), although the reference list in Elbaum et 
al. mistakenly describes Acalins work as an unpublished doctoral 
dissertation. Even a casual reading of these two studies reveals that 
they do not qualify as investigations of Reading Recovery inter- 
ventions. In fact, a closer reading of these two studies identifies 
serious problems. 



2. Evans unpublished doctoral dissertation 

Both the Internet letter and the Elbaum et al. meta-analysis use 
Evans’ unpublished doctoral dissertation (1996) as evidence that 
Reading Recovery provides no advantage over small group instruc- 
tion. Yet the Evans study is an unpublished qualitative doctoral 
dissertation designed to explore literacy acquisition and peer inter- 
action based on eight case study descriptions. The entire study 
consisted of eight children: four randomly assigned to Reading 
Recovery and four assigned to a small group intervention. The 
researcher was also the teacher of the small group intervention, 
and the Reading Recovery teacher was in the first months of her 
Reading Recovery training year. The researcher-small group leader 
“had taught young children in public elementary schools for seven 
years” (p. 29), while the Reading Recovery teacher’s experience 
“had primarily been in third through fifth grades” (p. 29—30) 
before beginning Reading Recovery training. The study was con- 
ducted between August and December, at the beginning of the 
Reading Recovery teacher’s training. 

Evans stated that the eight children had similar entry scores but 
did not provide data for selection or matching subjects. The sum- 
mary table of average pre- and post-test scores (p. 129) listed lower 
pre-test scores and larger standard deviations for the Reading 
Recovery treatment group across all six subtests (with the excep- 
tion of an equivalent pre-test score for text reading level, at 1.25). 
Yet Elbaum and colleagues reported a mean effect size for Evans’ 
study despite the lack of evidence of equivalence at pre-test. 



3. Acalin unpublished master s thesis 

The stated purpose of Acalins (1995) study was to compare the 
effectiveness of Reading Recovery to Project Read. Project Read is 
a remedial reading program “originally designed to be used in 
kindergarten through eighth grade” (p. 20). The results of this 
study are suspect because of the training provided. The specialists 
who provided the Project Read instruction “had received the full 
training in this program and were following the manuals, lesson 
by lesson, with minimal program adaptations” (p. 35). Yet the 
Reading Recovery treatment was not delivered by Reading 
Recovery teachers, but by special education teachers who had not 
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participated in the required Reading Recovery training. 
Descriptions of the instructional methods used for Reading 
Recovery in this study indicate wide discrepancies from published 
Reading Recovery procedures. 

Further, although Reading Recovery is a first-grade intervention, 
Acalin provided instruction to 66 subjects ranging from kinder- 
garten through fourth grade and included students who were 
already placed in a resource specialist program and identified as 
learning disabled. Only 8 of the 66 children were in the first 
grade, and four of these children were assigned to each approach. 

Even casual consumers of scientific research would wonder why 
the Evans and Acalin studies were considered to have met criteria 
for inclusion in a meta-analysis that purports to follow, in the 
words of Elbaum et al., “best practices for research synthesis” 

(2000, p. 606). 

A body of research supports 
one-to-one tutoring and 
indicates that it may be 
essential for children who are 
at high risk. 



Because Iversens study used a modified version of Reading 
Recovery, it seems the Internet letter authors are mixing two very 
different researchable issues: (1) standard Reading Recovery versus 
a small group intervention and (2) any one-to-one intervention 
versus a small group intervention. To examine Reading Recovery 
versus small group instruction, the standard program deserves to 
be studied. Therefore, the design of this study does not allow any 
comparisons of group instruction to standard Reading Recovery. 



4. Iversen study 

The Internet letter cites Iversens (1997) study to claim no advan- 
tage of one-to-one instruction over small group instruction. In this 
study, 75 first graders were grouped in triads (without random 
assignment). Each trio was taught by the same teacher, one indi- 
vidually and the others in a group of two, for a maximum of 60 
lessons. Iversen concluded there was no advantage of individual 
over small group instruction. Yet the Reading Recovery interven- 
tion in this study was far from standard Reading Recovery. There 
were major differences in the training model, the procedures relat- 
ed to selection of children, teaching procedures, and issues of 
implementation and evaluation practices. In addition, there are 
questions about design and methodology. 
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D. Summary 

Cost-effectiveness is a complex concept that cannot be oversimpli- 
fied. It is misleading to describe any program as expensive or not 
cost-effective without first identifying crucial information such as 
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which students the program targets and what results are sought 
relative to the performance of other students. Both long- and 
short-term benefits must be considered. 

In the Internet letter, concerns about Reading Recovery costs focus 
primarily on one-to-one instruction and the highly trained teach- 
ers Reading Recovery requires versus group instruction. Solid evi- 
dence (Dorn & Allen, 1995; Pinnell et al., 1994) supports the 
effectiveness of Reading Recovery versus small groups for lowest- 
performing first graders. Evidence that small group instruction is 
as effective with this group of learners is seriously suspect. 

Studies cited by Elbaum et al. provide virtually no evidence to 
support a change from one-to-one to small group instruction for 
the lowest-achieving first graders. The suggestion for Reading 
Recovery to change from one-to-one instruction is especially weak 
because there is documented evidence of effectiveness with hun- 
dreds of thousands of children. Certainly, researchers within and 
outside Reading Recovery should continue to study all possibili- 
ties, but the idea of change in group size needs a much stronger 
research base. A body of research supports one-to-one tutoring 
and indicates that it may be essential for children who are at high 
risk (Bloom, 1984; Juel, 1991; Wasik & Slavin, 1993). The sys- 
tematic nature of Reading Recovery instruction is based on a 
teachers detailed assessment and analysis of a child’s knowledge 
base and skills. The teaching is highly efficient because the teacher 
has this precise inventory of skills and strategies and is able to 
teach exactly what the child needs to know next. 

To use their own words, Elbaum and colleagues conclude, “In 
sum, the findings of this meta-analysis support the argument that 
well-designed, reliably implemented, one-to-one interventions can 
make a significant contribution to improved reading outcomes for 
many students whose poor reading skills place them at risk for 
academic failure” (2000, p. 617). 
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SEOON III 



READING RECOVERY USES 
STANDARD ASSESSMENT MEASURES. 



This section first describes An Observation Survey of Early Literacy 
Achievement (Clay, 1993a/2002) and its history and use in early 
literacy settings as well as in Reading Recovery. Second, Reading 
Recovery students’ performance in follow-up studies using norm- 
referenced tests is reported. 



A. Reading Recovery uses An Observation Survey of 
Early Literacy Achievementy a standard measure 
developed in research studies, with qualities of 
sound assessment instruments with reliabilities and 
validities and discrimination indices. 



1. The Observation Survey was developed to meet the 
unique need to assess emergent literacy in young 
children. 

In the 1960s there were few studies of literacy acquisition or credi- 
ble theories of literacy development using close observation over 
time during the first year of school. Marie Clay was committed to 
rationality and scientific methodology; the research designs she 
used and the standard measures she developed followed rigorous 
standards of research. Since the 1960s, Clay has engaged in more 
than 40 years of in-depth research and analysis of evidence to con- 
struct her theory and validate measures that describe the range of 
differences and change over time among the lowest-achieving stu- 
dents (Clay, 2001). These studies, discussed in Observing Young 
Readers (Clay, 1982), extend or verify elements of her theory. 



Clay sought to understand why children fail to realize their learn- 
ing potential and to describe the course of literacy development 
and the different paths students might take. The research methods 
she used produced assessment tools that have high construct and 
face validity and high reliability measured in large-scale studies. 
Measurement error within these tasks is greatly reduced with indi- 
vidual administration and with standardized administration proce- 
dures (see An Observation Survey of Early Literacy Achievement 
Clay, 1993a/2002).^ 



3 Until the Observation Survey was published in 1993, it was known as the Diagnostic 
Survey (Clay, 1985). 
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The Observation Survey 
adheres to characteristics 
of good measurement 
instruments, namely, a 
standard task, a standard 
way of administering the 
task, ways of knowing about 
reliability of observations, 
and a real-world task that 
establishes validity of 
the observation. 



Tasks included in the Observation Survey incorporate both open 
and closed tasks. They allow for observation of emerging, tentative 
behaviors to detect the variability of individual paths to literacy 
achievement. “Powerful statistical analyses have shown that these 
procedures, which permit more detailed recording of individual 
responses than a normative test, nevertheless have proved to be 
sound measurement devices” (Clay, 1982, p. 6). 

A particular strength for Reading Recovery in the United States is 
that parallel instruments have been developed for the Spanish lan- 
guage, and they have been subjected to the same rigorous analyses 
for reliability and validity (see Escamilla, Andrade, Basurto, & 
Ruiz, 1996). 

2. The Observation Survey comprises systematic and con- 
trolled observation measures to assess young childrens 
literacy knowledge and to detect evidence of progress in 
the early stages of literacy learning. 

The six tasks of the Observation Survey have been widely used 
with the five- to seven-year-old age group; they are not in-house 
instruments used only for Reading Recovery assessments. Users of 
the Observation Survey include classroom teachers, teachers work- 
ing individually with children having temporary difficulties with 
literacy learning, administrators who want accounts of individual 
progress of children across time, and researchers probing how 
young children learn about literacy. 

These controlled observation tasks have been widely used in litera- 
cy research. They 

can feed data into analyses of researchers, and best of all, 
they can provide evidence of learning on repeated meas- 
urements of tasks the child is actually undertaking in the 
classroom. In every way the information that is gathered in 
systematic observation reduces our uncertainties and improves 
our instruction. (Clay, 1993a/2002, p. 2) 

The Observation Survey adheres to characteristics of good meas- 
urement instruments, namely, a standard task, a standard way of 
administering the task, ways of knowing about reliability of 
observations, and a real-world task that establishes validity of the 
observation. 

The Observation Survey is comprised of six literacy tasks with 
established validity and reliability (see Clay, 1993a/2002). 

• Letter Identification (to identify known letters and pre- 
ferred mode of identification) 
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• Word Test (to determine whether the child is building 
reading vocabulary) 

• Concepts About Print (to find out what the child has 
learned about the way spoken language is put into print) 

• Writing Vocabulary (to find out whether the child is 
building a writing vocabulary) 

• Hearing and Recording Sounds in Words (to assess 
phonemic awareness by responses to sound-letter associa- 
tions) 

• Text Reading (to determine appropriate level of text diffi- 
culty and to record, using a running record, what the 
child does when reading continuous text) 

The six tasks can be justified by theories of measurement, and they 
take other theories into account (from the psychology of learning, 
developmental psychology, studies of individual differences, and 
theories about social factors and the influences of contexts on 
learning). Stanines and reliability/ validity data are provided for five 
of the tasks within the Observation Survey. 

In the 2002 edition of the Observation Survey, new norming data 
are provided. New norms will be established for the United States 
during the 2002—2003 academic year. 

Many of the tasks in the Observation Survey are similar to tasks in 
other widely used standardized and norm-referenced tests. 

Informal reading inventories are used to determine the appropriate 
text level for a student s instruction, to monitor progress of indi- 
vidual students, and to obtain pre- and post-test scores in the pri- 
mary grades. The tasks of the informal reading inventories (word 
lists and text reading) are similar to the Word Test and Running 
Records of Text Reading in the Observation Survey. 

Word identification tasks are common in both standardized and 
norm-referenced tests (Woodcock-Johnson III, Slosson Oral 
Reading Test, Qualitative Reading Inventory-3, Iowa Test of Basic 
Skills, and others). Reading passages organized along a gradient of 
difficulty are also prevalent in many tests (Basic Reading 
Inventory, Qualitative Reading Inventory-3, Gates-MacGinite 
Reading Tests, and Iowa Test of Basic Skills among others). 

Many tests assess beginning readers’ phonemic awareness. The 
Hearing and Recording Sounds in Words task of the Observation 
Survey requires the student to articulate words slowly, supplying 
the letter or letters associated with the phonemes within each word 
of the dictated sentences. 
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3. Reading Recovery analyses of text reading levels provide 
descriptive data of behavior on a scale of relative diffi- 
culty and provide data about change across time. 

The text reading measure used within the Reading Recovery pro- 
gram uses texts from the Scott Foresman Reading Systems Special 
Practice Books (1979). To standardize the administration of the 
text reading measure, brief story introductions were produced and 
a standard format of administration was established. In a 1987- 
1988 study, Ohio State University researchers piloted texts and 
procedures. Changes were made and a larger-scale sampling and 
comparison was completed to test these materials against a previ- 
ously used testing program, with resulting .85 reliability. 

In 1990—1991, a random sample of 155 urban kindergarten and 
first-grade children were sampled using the Diagnostic Survey 
tasks'^ (Clay, 1985) and the Reading Recovery text reading materi- 
als. An analysis of the text reading materials was completed on the 
first graders in the study (n=96) to determine the reliability of the 
scale using a Rasch rating scale analysis (Wright, Linacre, & 

Schulz, 1989). This analysis showed that the text reading scale had 
a reliability of .83 (person) and .98 (item). When the text reading 
measure was combined with two other tasks (a measure of print 
concepts and phonemic representation), the power of the assess- 
ment increased. It verified that while the text reading measure 
does not provide an equal interval scale, the item difficulty scale 
itself across these three measures is robust and highly reliable (scale 
formula r=.99). 

Reading Recovery analyses of text reading levels provide descrip- 
tive data of behavior on a scale of relative difficulty, and they pro- 
vide data about change across time. These analyses are appropriate 
assessment and measurement techniques commonly used in the 
educational measurement field to validate such observation data as 
ordinal information, time series samples, and more. These tech- 
niques to transform observations into measures date back to the 
turn of the century (Thorndike, 1904; Thurstone, 1925; Wright 
& Linacre, 1989) and have since been perfected by advanced sta- 
tistical procedures (Rasch, I960, 1980). 

4. Reading Recovery appropriately uses the Observation 
Survey in pre-treatment and post-treatment analyses of 
childrens progress. 

With the exception of the Letter Identification task and the 

^ The Diagnostic Survey (1985) was published as part of The Early Detection of 
Reading Difficulties: A Diagnostic Survey With Reading Recovery Procedures. It was later 
renamed The Observation Survey of Early Literacy Achievement and published 
separately in 1993 and 2002. 
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Writing Vocabulary tasks, all sub-tests of the Observation Survey 
have three or more alternate forms. Data collected annually using 
the standard assessment measures offer a variety of ways to verify 
the quality of decisions made about selection of students and out- 
comes achieved. The standard measures of the Observation Survey 
are used to select the lowest literacy achievers for service and to 
make reliable decisions about student progress. 

Reading Recovery children begin first grade with much lower 
scores than their peers; yet children who meet discontinuing crite- 
ria reach approximate parity by the end of first grade. Discontinu- 
ing decisions are also corroborated by external factors such as 
classroom teacher perceptions and low rates of retention and 
placement in special education. The methodology must fit the 
research questions and issues being addressed. The goal remains 
the same: select the lowest-achieving students, provide early inter- 
vention that reduces the need for further remediation, and moni- 
tor change over time using standard measures appropriate for the 
beginning reader. 



B. Reading Recovery students perform well on norm- 
referenced tests. 

The Internet letter suggests use of norm-referenced tests that are 
widely available and commonly used in reading intervention 
research. Although traditional standardized tests may yield valid 
comparisons of mean scores derived from groups of students who 
are already reading, they are not sensitive to variability in emerging 
knowledge. These tests are not useful as baseline measures for 
assessing change over time in individual young learners. For this 
reason, such tests are not used for selection into Reading Recovery. 

Some studies, however, have examined Reading Recovery children 
in Grade 1 (see for example Pinnell et ah, 1994) using such assess- 
ments as the Slosson Oral Reading Test and the Woodcock- 
Johnson III. Many more studies have used standardized independ- 
ent measures and state assessments to explore Reading Recovery 
childrens performance in subsequent grades. 

In California, a study of four cohorts of former Reading Recovery 
students (760 students in Grades 2, 3, 4, and 5) revealed that 
68-85% (percentages varied by year and by group) scored at 
Stanine 4 or above on both of two high-profile standardized tests, 
the Iowa Test of Basic Skills and Stanford Achievement Test 9 
(Brown et ah, 1999). 



Two longitudinal studies in Texas used the Gates-MacGinitie 
Reading Test and the Texas Assessment of Academic Skills to 
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explore subsequent performance of Reading Recovery children 
compared with a random sample of their classroom peers (Askew 
et al., 2002). Findings showed that 80-85% of the former discon- 
tinued Reading Recovery children passed the fourth-grade Texas 
Assessment of Academic Skills reading test and 90-93% passed 
the writing test. Annual progress on the Gates-MacGinitie 
Reading Test closely paralleled the progress of the random sample. 



Although the Internet 
letter suggests use of norm- 
referenced tests instead of the 
Observation Survey, these 
traditional standardized tests 
are not useful as baseline 
measures for assessing change 
over time in individual 
young learners. 



An Indiana study of the performance of former successful Reading 
Recovery children on the Gates-MacGinitie Reading Test found 
that 86% of those children currently in second grade scored with- 
in the average range of scores as established by a random sample 
group; 84% currently in third grade and 80% in fourth grade 
scored within the average range (Schmitt & Gregory, 2001). 

Scores on the third-grade Comprehensive Test of Basic Skills-5/ 
Terra Nova Form B assessments administered statewide approxi- 
mated a normal distribution for former Reading Recovery children 
with a mean at the 45th percentile. 



C. Summary 

Reading Recovery uses the measurements published in An 
Observation Survey of Early Literacy Achievement (Clay, 
1993a/2002), a standard measure developed in research studies 
with qualities of sound assessment instruments with reliabilities 
and validities and discrimination indices. The survey adheres to 
characteristics of good measurement instruments: namely, it is a 
standard task, there is a standard way of administering the task, 
there are ways of knowing about reliability of observations, and it 
is a real-world task that establishes validity of the observation. 

Reading Recovery appropriately uses the Observation Survey in 
pre-treatment and post-treatment of childrens progress. vMthough 
the Internet letter suggests use of norm-referenced tests instead of 
the Observation Survey, these traditional standardized tests are not 
useful as baseline measures for assessing change over time \n young 
learners. For this reason, these tests are not used for selection of 
children for Reading Recovery. When children have developed 
more literacy skills by the end of Grade I and in later grades, 
standardized measures are often used to examine subsequent 
literacy achievement for Reading Recovery students (Askew et al., 
2002; Brown et al., 1999; Pinnell et al., 1994; Schmitt & 

Gregory, 2001). 
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SECTION IV 



CHANGE IS AN INTEGRAL PART OF 
THE READING RECOVERY DESIGN. 



Change is an integral part of the Reading Recovery design. Nobel 
Prize v^inner Kenneth Wilson wrote, “Reading Recovery offers 
U.S. education its first real demonstration of the power of a 
process combining research, development (including ongoing 
teacher education), marketing, and technical support in an orches- 
trated system of change” (Wilson & Daviss, 1994, p. 76). 

Reading Recovery is a dynamic program that changes in response 
to ongoing research. Theoretical constructs, teaching practices, and 
implementation are constantly examined in search of the most 
effective procedures (Jones & Smith-Burke, 1999). An examina- 
tion of the evolving works of Marie Clay provides evidence of 
change over time. 

This section highlights issues of change and constancy that pro- 
vide both flexibility and stability for Reading Recovery at the 
national, school district, and local school levels. This section is 
organized into two parts: 

A. Reading Recovery's built-in mechanisms for change 
including the central document for theory and practice, 
the internal processes of data collection and reportings 
the ongoing professional development process, the stan- 
dards and guidelines, and the international network 

B. Areas of change recommended by the Internet letter, 
especially the inclusion of explicit instruction in phon- 
ics and phonemic awareness 



''Reading Recovery offers 
U.S. education its first real 
demonstration of the power 
of a process combining 
research, development 
(including ongoing teacher 
education), marketing, and 
technical support in an 
orchestrated system of 
change.” 

(Wilson and Daviss, 1 994) 



A. Reading Recovery has built-in mechanisms for 
change. 

Change in Reading Recovery is not left to chance. Several systems 
ensure careful and responsible responses to patterns of change and 
constancy. 



1 . The Guidebook reflects changes in theory and practice. 



Patterns of change are evident in the evolving editions of Reading 
Recovery's central program document, Reading Recovery: A 
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The Guidebook is a central 
and constant resource for 
teachers not only during 
professional development 
sessions but as they do their 
daily teaching. 



Guidebook for Teachers in Training (Clay, 1993b). Originally pub- 
lished as Early Detection of Reading Difficulties (1979/1985), it was 
thoroughly revised and redded in 1993. An examination of these 
books reveals significant additions over years of development, 
including the following: 

• more intensive attention to and detailed description of 
the role of phonemic awareness; 

• explicit directions for teachers in helping children use 
letter-sound relationships, more phonemic awareness, 
and phonics; 

• more deliberate focus on comprehending strategies dur- 
ing the reading of a new book; 

• differentiation between the way the teacher supports 
children during the reading of a new text and the role of 
familiar reading; and, 

• more information on how to teach for fluency and 
phrasing. 

The Guidebook is a central and constant resource for teachers not 
only during professional development sessions but also as they do 
their daily teaching. Changes in the Guidebook result from a care- 
ful examination of current research across a wide range of disci- 
plines and perspectives and are carefully examined and acted upon 
each year by all persons involved in Reading Recovery. Professional 
development for purposes of increasing teacher skill and updating 
practice is a requirement for Reading Recovery teachers as long as 
they are involved in Reading Recovery. 



2. Data collection and analysis guide changes. 

Internal Reading Recovery processes use data collection and analy- 
sis to inform changes in implementation at local, state, and 
national levels. Reading Recovery teachers, teacher leaders, and 
administrators at every training site systematically collect and 
report data on every child to the National Data Evaluation Center 
located at The Ohio State University. The center reports aggregat- 
ed data at the school level, the school system level, the training site 
level, and the university training center level. Data collected allow 
school personnel to analyze results and improve implementation 
efficiency and effectiveness at all levels. 

A partial list of data collected follows: 

• status (outcome) categories of children: discontinued, 
recommended, incomplete, moved, none of the above 

• number of lessons for individual children 
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• number of weeks for individual children 

• reasons for missed lessons for individual children 

• years of experience for teachers 

• number of retentions in first grade 

• number of special education referrals and placements in 
first grade 

• level of Reading Recovery coverage in the school (percent 
of lowest-achieving children served) 



Analysis of these data has revealed important findings about 
Reading Recovery implementation at the national level. Among 
the findings, 



• The level of Reading Recovery teacher cov- 
erage in schools is correlated with program 
success. 

• Time is an important factor in program 
efficiency. Data are available for examina- 
tion of time related to length of programs, 
missed lessons, and more. 

• Teacher factors are also related to program 
outcomes. For example, trained and experi- 
enced teachers on average have more suc- 
cessful student outcomes than teachers-in- 
training. 

Local schools, districts, and university training 
centers use outcome data and implementation 
data to address program questions. In addition, 
observational and anecdotal data often lead to 
increased emphasis on implementation issues at 
the national level. For example, national confer- 
ences and publications focus on the key role local 
administrators play in creating an effective and 
efficient implementation. 




Ongoing professional development ensures incorporation of 
new research and understanding of its applicability to the 
Reading Recovery lesson. A one-way mirror, a required part of 
Reading Recovery professional development, enables teachers 
to observe, reflect on, and discuss Reading Recovery lessons 
with a teacher leader or university trainer. 



3. Ongoing professional development disseminates changes. 

Ongoing professional development ensures incorporation of new 
research and understanding of its applicability to the Reading 
Recovery lesson. Reading Recovery’s professional development 
builds teacher skills and provides continuous updating so that 
trained Reading Recovery teachers incorporate changes as they 
occur. In fact, Hermann and Stringfield (1997) noted Reading 
Recovery’s high-quality professional development: 'As schools 
systematize and create more opportunities for serious staff devel- 
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opment, the thoroughness of the Reading Recovery model seems 
to be well worth emulating.” 

Ongoing professional development occurs at three levels: universi- 
ty trainers, site-based teacher leaders, and school-based teachers. 
Ongoing development is required for all Reading Recovery profes- 
sionals throughout their involvement in Reading Recovery. 

This continuing professional development across the entire 
Reading Recovery network is an important mechanism in the 
change process. Reading Recovery professionals receive up-to-date 
knowledge of new developments in the program. This self-renewal 
system accommodates changes that result from sound research and 
from carefully monitored developments. 



‘‘As schools systematize 
and create more 
opportunities for serious 
staff development, the 
thoroughness of the 
Reading Recovery model 
seems to be well worth 
emulating.” 

(Hermann and Stringfieldy 

1997) 



4. Published standards and guidelines reflect changes. 

Patterns of change and constancy are also evident in the Standards 
and Guidelines of the Reading Recovery Council of North America. 

It is well known that the effectiveness and efficiency of any inter- 
vention are tied directly to issues of implementation within a 
school or system. Most recently revised in 2001, the standards 
and guidelines are intended to protect the investment of adopters 
who are responsible for the establishment and maintenance of 
Reading Recovery. 

Standards and guidelines are not arbitrary: they are based on 
underlying rationales supported by research on effective practices, 
national data evaluation, and research and observational data from 
the field. Changes are made as needed. Since 1993 there have been 
three editions of the standards and guidelines. 

5. An international network of Reading Recovery trainers 
works to ensure the dynamic nature of the intervention. 

An international network of Reading Recovery trainers (including 
faculty members from 23 universities in the United States) is 
responsible for guiding ongoing development and research efforts 
in Reading Recovery. The university trainer must have knowledge 
of what it means to bring about cycles of change in practice, but 
in ways that are consistent with the academic theories which sup- 
port the program. Trainers bring several areas of expertise together 
in an ongoing way as Reading Recovery is problem-solved into 
educational settings. The university trainers role is to think inte- 
gratively about theory, bringing diverse areas of current theoretical 
and practical knowledge together into working relationships. 
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B. Reading Recovery responds to Internet letter change 
recommendations. 

The Internet letter recommends three areas of change for Reading 
Recovery: increased group size, use of standardized outcome meas- 
ures, and explicit instruction in phonics and phonemic awareness. 

1. Increased group size 

Internet letter arguments for change in group size are weak: To 
deny children effective one-to-one intervention based on research 
cited in the Internet letter would be irresponsible. The cost-effec- 
tiveness section of this report reviews substantial research evidence 
on the efficacy of one-to-one tutoring with the lowest-performing 
students (see pages 31 to 4l). 



2. Use of standardized outcome measures and continuous 
progress monitoring 

Section III on assessment measures reviews the Observation Survey 
of Early Literacy Achievement (Clay, 1993a/2002), a standard meas- 
ure developed in research studies and used by Reading Recovery 
professionals. The Observation Survey includes the quality of 
sound assessment instruments with reliabilities, validities, and dis- 
crimination indices. In addition, research reviewed in Sections I 
and III demonstrates that Reading Recovery students perform well 
on norm-referenced tests after the intervention year (Askew et al., 
2002; Brown et. al., 1999; Pinnell et al., 1994; Schmitt & 
Gregory, 2001). 



Children are shown how to 
use letter-sound relationships 
to solve words in reading 
and writing and how to use 
structural analysis of words 
and learn spelling patterns. 



3. Explicit instruction in phonics and phonemic awareness 

The Internet letter claims that Reading Recovery does not include 
explicit instruction in phonics and phonemic awareness. Yet any 
astute observer of a Reading Recovery lesson would recognize the 
explicit teaching of letters, sounds, and words. Children are shown 
how to use letter-sound relationships to solve words in reading and 
writing and how to use structural analysis of words and learn 
spelling patterns. (For a detailed description of instruction in 
phonics and phonemic awareness within the Reading Recovery les- 
son, see Pinnell, 2000). 

The Internet letter uses two studies (Iversen & Tunmer, 1993; 
Morris et al., 1990) to support this claim. The analyses that follow 
demonstrate the difficulties associated with using these studies as 
evidence. 
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Interestingly, the increase 
in phonemic awareness 
and phonics that Iversen 
included in training of her 
experimental group had 
already been included in 
Reading Recovery programs 
around the world. 



a. Iversen and Tunmer (1993). Iversen conducted a study 
that included two groups of Reading Recovery teachers-in- 
training. As the sole staff developer, she taught both groups, 
giving to one group the charge to implement an extra few 
minutes of phonological awareness training. (See page 18 
for additional information regarding this study.) 

Iversen herself had been trained as a teacher leader during 
the early years of Reading Recovery in New Zealand but 
was no longer teaching Reading Recovery and was not 
attending continuing professional development to receive 
program updates. Therefore, she was unaware that impor- 
tant changes to increase emphasis on phonemic awareness 
and phonics had already been incorporated in Reading 
Recovery. Thus, the Reading Recovery training provided to 
the traditional Reading Recovery group in effect deprived 
them of information about newer teaching practices. 

Even so, results indicated that both Reading Recovery 
groups outperformed the control group on all measures — 
including measures of phoneme deletion and phoneme 
segmentation. The group with increased emphasis on 
phonemic awareness had programs that were shorter than 
Iversens traditional group. Interestingly, the increase in 
phonemic awareness and phonics that Iversen included in 
training of her experimental group had already been 
included in Reading Recovery programs around the world. 

b. Morris, Tyner, and Perney (2000). This is the second 
study cited in the Internet letter in support of explicit 
phonics. This study looked at some alternative staffing, 
training, and instructional approaches to early interven- 
tion. Morris and his colleagues did place a greater empha- 
sis on isolated approaches to early intervention, but they 
also modeled more than three-quarters of the lesson format 
on Reading Recovery. The results indicated that students 
who participated in their First Steps program made better 
progress than a matched group of low students in nonpar- 
ticipating comparison schools. This study was not designed 
to compare results against Reading Recovery or to isolate 
the contribution of a particular form of word study in rela- 
tion to other program components. The claim in the 
Internet letter that “the addition of an explicit component 
addressing spelling-to-sound patterns was highly effective” 
(paragraph 7) seems questionable given that First Steps 
students received tutoring for the entire school year, aver- 
aging 91 lessons per student. The Morris, Tyner, and 
Perney study provides no demonstrated evidence that com- 
ponents should be added to Reading Recovery lessons. 
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C. Evidence shows that Reading Recovery teaches 
phonemic awareness and phonics. 

A study by Stahl et al. (1999) demonstrates that Reading Recovery 
students do, in fact, perform well on tests of phonemic awareness 
and phonological coding. The study was conducted to determine 
whether techniques used in Reading Recovery lessons promoted 
progress in the metalinguistic areas of phonemic awareness and 
phonological recoding. A total of 30 at-risk first-grade students 
were rank ordered. The lowest- achieving children (n=l 1) were 
entered into Reading Recovery. A control group of 19 subjects eli- 
gible for Reading Recovery was formed. Measures used included 
the Observation Survey, particularly the Letter Identification and 
Hearing and Recording Sounds in Words tests; the Pseudoword 
Learning Test; and Yopp-Singer Test of Phoneme Segmentation, a 
test that measures students’ ability to hear and articulate sequen- 
tially the separate sounds in 22 words. 

Reading Recovery students made significantly greater improve- 
ment than the control group on measures of phonological process- 
ing. Students who successfully completed Reading Recovery les- 
sons demonstrated strategies similar to children in the alphabetic 
stage by the 16th week of Grade 1. These students were using 
strategies that were similar to those used by normally achieving 
first graders. This study suggested that Reading Recovery children 
acquire phonological awareness and phonological recoding with 
Reading Recovery lessons without additional lesson components in 
phonological processing. The inclusion of all Reading Recovery 
participants and the utilization of measures other than Clays 
responded to methodological concerns raised in other reports. 

Other researchers have also stressed that Reading Recovery teaches 
phonics. Writing in 1990 about her observation of Reading 
Recovery, Marilyn Adams said, ‘‘The importance of phonological 
and linguistic awareness is also explicitly recognized” (p. 420). 
Adams describes Reading Recovery as one of several programs that 

“are designed to develop thorough appreciation of phonics On 

the other hand, none of these programs treats phonics in a vacu- 
um” (p. 421). 



D. New Zealand researcher responds to the Internet 
letter. 

Authors of the Internet letter point to concerns about Reading 
Recovery in New Zealand where Reading Recovery is a widely 
used program for lowest-performing beginning readers. As evi- 
dence of dissatisfaction, the letter cites a unanimous recommenda- 
tion from a report of the Literacy Experts Group. Two signatories 
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''Most of this debate is about 
some researchers talking to 
some other researchers, with 
very little buy in from the 
teachers who implement 
Reading Recovery, or 
those with experience of 
translating research findings 
into effective classroom 
programmes. And, there is 
a huge gulf between a 
research study and a 
programme that works in 
a classroom,” 

(Crofty 2002) 



of the Internet letter were from the 10-member group. In response 
to an article in the New Zealand newspaper Education Weekljy 
Cedric Croft, chief researcher from the New Zealand Council for 
Educational Research, wrote 

It would be regrettable if. ..recommendations from the 
1999 Literacy Experts Group, was construed as meaning 
that this group was among those attacking Reading 
Recovery. Because it wasn’t, even though two of its 10 
members are among a group of 32 predominantly 
American reading researchers reported as having ‘recently 
circulated a letter criticizing Reading Recovery to members 
of Congress and through the media.’. . . 

One recommendation [that Reading Recovery place 
greater emphasis on explicit instruction in phonological 
awareness] from the Literacy Experts Group was reported 
as being unanimous but so were the other 17! 

Another recommendation from the Literacy Experts 
Group is worth quoting too. ‘We recommend continued 
government funding of Reading Recovery as a national 
systematic programme, providing New Zealand research is 
carried out to determine whether it is as effective as it 
could be’... 

Most striking, however, is the clear message that most of 
this debate is about some researchers talking to some other 
researchers, with very little buy in from the teachers who 
implement Reading Recovery, or those with experience of 
translating research findings into effective classroom pro- 
grammes. And, there is a huge gulf between a research 
study and a programme that works in a classroom. This is 
one area where the four New Zealand signatories of the 
U.S. [Internet] letter are out-of-step with many other read- 
ing researchers and literacy educators in New Zealand. 

(2002, p. 2-3) 

E. Summary 

The change process in Reading Recovery is ongoing and based on 
careful testing of components over time. Changes are not made in 
response to one or two studies or to insufficient evidence. But 
changes have been documented in the past and are expected to be 
part of the future. The Internet letter has not provided compelling 
evidence for change, nor has it proven that Reading Recovery does 
not change or incorporate independent research results. 

One of the key changes recommended by the Internet letter is 
explicit instruction in phonics and phonemic awareness. Reading 
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Recovery has a strong phonemic awareness component including 
the explicit teaching of letters, sounds, and words. Reading 
Recovery also shows children how to use letter-sound relationships 
to solve words in reading and writing, how to use structural analy- 
sis of words, and how to learn spelling patterns (Pinnell, 2000). A 
study by Stahl et al. (1999) demonstrated that Reading Recovery 
students do in fact perform well on standardized tests of phonemic 
awareness and phonological coding. Researcher Marilyn Adams 
wrote this about Reading Recovery: “The importance of phono- 
logical and linguistic awareness is also explicitly recognized” 

(1990, p. 420). 

Programs that focus on items of knowledge in prepared curricular 
sequences may teach the targeted objectives effectively, but they 
have provided no evidence of successfully closing the gap between 
at-risk learners and their classmates, nor have they provided evi- 
dence of developing abilities to learn that continue beyond the life 
of the programs. Reading Recovery’s power lies in the fact that it is 
preventative in nature. Childrens learning accelerates while they 
are being tutored in Reading Recovery, and they develop a system 
for learning that is self-extending. They continue to learn more 
about reading and writing as they engage in meaningful literacy 
activities in classrooms. 

Change is integral in Reading Recovery. Mechanisms for change 
are built into the Reading Recovery design and are reflected in the 
Guidebook, the internal processes of data collection and reporting, 
the ongoing professional development process, the standards and 
guidelines, and the international network. 

Reading Recovery is one of the few highly dynamic programs in 
existence. Across the United States, 23 universities participate in 
the implementation and further development of Reading 
Recovery. Worldwide, university faculty members collaborate to 
examine research evidence and suggest research priorities. 

One of the difficulties of United States school reform is the con- 
stant swing as we trade and discard various approaches. Successful 
approaches like Reading Recovery need to be secure so they can 
continue to develop deliberately and in slow increments supported 
by research. Only with this kind of thoughtful approach will we 
build successful programs that really work for our students. 



One of the difficulties of 
United States school reform 
is the constant swing as we 
trade and discard various 
approaches. Successful 
approaches like Reading 
Recovery need to be secure 
so they can continue to 
develop deliberately and in 
slow increments supported 
by research. 
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APPENDIX A 



INTERNET LETTER 



EVIDENCE- BASED RESEARCH 
ON READING RECOVERY 

We are an international group of researchers who study reading development and interventions 
with struggling readers. This letter responds to a number of questions that have been raised by educators, 
policymakers, and parents about the effectiveness of Reading Recovery, a tutoring program designed for 
struggling first grade students. We hope the following summary analysis will be helpful to those who are 
considering the most effective ways to help struggling students become proficient readers. 

These are not isolated opinions and the findings here are summaries of several peer-reviewed 
studies and syntheses of research on Reading Recovery. However, it is not our goal to discredit Reading 
Recovery, but as with any other program, outline its weaknesses to suggest how it can be improved. We 
believe this should be done for any program that is widely used to address reading difficulties. 

Reading Recovery is not successful with its targeted student population, the lowest performing stu- 
dents. There is little evidence to show that Reading Recovery has proved successful with the lowest per- 
forming students. Reading Recovery targets the lowest 10-20 percent of first graders who have the prereq- 
uisite skills for Reading Recovery. While research distributed by the developers of Reading Recovery indi- 
cates a positive effect of the program, analyses by independent researchers have found serious problems 
with these conclusions. Studies conducted by researchers associated with Reading Recovery typically 
exclude 25-40% of the poorest performing students from the data analysis. In contrast, the studies funded 
by the National Institute for Child Health and Human Development (NICHD) and the Office of Special 
Education Programs (OSEP) in the Department of Education never purposely exclude a child. The data on 
efficacy is based on all those who are enrolled and available for follow- up. This is known as an ‘‘intent to 
treat” approach, which is standard for any evaluative research. Reading Recovery’s “in-house research” 
does not follow an “intent to treaf’ approach. In fact, for the poorest readers, empirical syntheses of “in- 
house” and independent studies indicate that Reading Recovery is not effective. In Elbaum et al. (2000), 
the gains for the poorest readers instructed with Reading Recovery were almost zero. There is also evi- 
dence that students who do complete the Reading Recovery sequence in first grade lose much of their 
gains, even in the 65-75% of better students who finish the program (Hiebert, 1994; Shanahan & Barr, 
1995; Snow, Burns, & Griffin, 1998; Tunmer & Chapman, in press b). A recent study by a group from 
New Zealand (Chapman, Tunmer, & Prochnow, 2001) shows that students in Reading Recovery may expe- 
rience problems with self-esteem when they do not perform well. One of the authors. Chapman, stated in 
an interview with a New Zealand newspaper (The Press, November 1, 1999) “Students actually declined in 
self-esteem throughout the course of the program and continued to show no acceleration or improvement 
in the period following the programme.”(See also Tunmer & Chapman, in press a). 

Reading Recovery is not a cost effective solution. Even if it were maximally effective, Reading 
Recovery is not cost effective because the developers require one-to-one interventions by highly trained 
teachers. An analysis by Hiebert (1994) found that Reading Recovery was very expensive, costing over 
$8,000 per student, reflecting in part the costs of training. But Elbaum et al. (2000) found that students 
who participated in Reading Recovery did not outperform students who were provided one-on-one reading 
instruction by trained volunteers. At least two studies have compared Reading Recovery in a one-to-one 
grouping with a modified version of “Reading Recovery” administered to a small group (by definition this 
can’t be Reading Recovery; Evans, 1996; Iversen, 1997). There was no advantage of one-to-one instruc- 
tion over small group instruction. There are other first grade programs that are demonstrably efficacious, 
impact more students because they do not require T. 1 tutoring, are easier to implement, and do a better job 
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than Reading Recovery of improving student reading skills because they do not drop students (Snow et al., 
1998; Torgesen, 2000). 

Altogether, several studies indicate that teacher: student groupings of 1:3 work as well as group- 
ings of 1:1 (Elbaum et al., 2000). Many of the current NICHD and OSEP pullout interventions utilize 
group sizes of 1:3 and higher. Thus, solely by virtue of the number of students who can be reached, 
Reading Recovery is at least 200% more expensive than other first grade interventions. Reading Recovery 
specifically states that it is not a program for groups, but provides little empirical support for this philoso- 
phy. This philosophy is inconsistent with the research on early intervention. 

Reading Recovery efficacy studies do not use standard assessment measures. Most evaluations are 
restricted to the Reading Recovery developers’ own, nonstandard measures. These same measures are used 
to determine which students will be considered as part of the sample (continued versus discontinued stu- 
dents). Thus, outcomes are inflated and unconvincing to the research community. The primary outcome 
measure used by Reading Recovery “in-house” researchers that has shown the largest effect is an assess- 
ment of “text reading” developed by the authors. However, even Reading Recovery specialists acknowl- 
edge that “The text reading measure is not an equal interval scale, that is, there are smaller differences in 
the beginning levels than at upper levels. For beginning readers, it is necessary to look at the reader’s 
progress in more detail” (Askew et al., 1998, p.lO). Obvious candidates would involve continuous progress 
monitoring as implemented in numerous research studies and norm referenced tests that are widely avail- 
able and commonly used in reading intervention research. With use of standard measures like those imple- 
mented by independent researchers, student performance could be compared across studies, permitting cal- 
culation of response to instruction based on the number of hours of instruction across interventions (see 
Torgesen, 2000). 

Reading Recovery does not change by capitalizing on research. Reading Recovery developers have 
been and continue to be resistant to integrating the findings of independent, scientifically based reading 
research into their program and making it more cost effective. The failure to attend to research in modify- 
ing the program is its major downfall. The lack of efficacy of Reading Recovery with the poorest readers 
is not surprising given the research base that highlights the importance of explicit teaching of phonics for 
this group. Reading Recovery teaches phonics, but the instruction is not sufficiently explicit. A common 
finding in research on Reading Recovery is that those students who do not respond are weak in phonologi- 
cal awareness (Snow et al., 1998; Tunmer & Chapman, in press b). In fact, research by New Zealand 
researchers Iverson and Tunmer (1993) in which an explicit phonics component was added to a standard 
Reading Recovery intervention reduced the time required to complete the program by about 30%. Morris, 
Tyner, and Perney (2000) found that a reading program constructed like Reading Recovery with the addi- 
tion of an explicit component addressing spelling-to-sound patterns was highly effective, even with those 
students most at risk. 

Reading Recovery has been independently evaluated in New Zealand, the country in which it was 
developed. These researchers, who have cosigned this letter, asked that this summary be included: 

“In New Zealand, where Reading Recovery was developed, the programme has been independ- 
ently examined on two occasions. Both studies found shortcomings. In essence, the programme is 
failing to meet the claims regarding its objectives and success. Senior Reading Recovery adminis- 
trators have also overtly blocked attempts by graduate students to independently examine aspects 
of Reading Recovery. The New Zealand Ministry of Education has stated that because of copy- 
right issues, the Ministry is unable to make changes to the program. Despite strong evidence in 
New Zealand, Australia, and the US that changes are needed to make Reading Recovery more 
effective, Reading Recovery leaders do not seem willing to incoi*porate the findings of such 
research to make the programme more effective. There is and has been considerable debate about 
the efficacy of Reading Recovery in New Zealand; this debate is indicative of an increasing dis- 
satisfaction among researchers and some educators about the nature of the Reading Recovery pro- 
gramme. Finally, the Ministry of Education commissioned a report from the “Literacy Experts 
Group”, released in 1999. Included in this report was a recommendation, unanimously agreed to 
by experts from the full spectrum of views on reading: “We recommend that Reading Recovery 
place greater emphasis on explicit instruction in phonological awareness and the use of Spelling- 
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to-sound patterns in recognizing unfamiliar words in text.” This recommendation has not been 
adopted by Reading Recovery.” 

There are three additions that would impact positively the number of students who benefit from 
Reading Recovery, their rate of progress, and reduce costs: (1) increased group size; (2) explicit instruction 
in phonics and phonemic awareness; and (3) use of standardized outcome measures and continuous 
progress monitoring. These additions have been ignored despite research summarized in the National 
Research Council report, Preventing Reading Difficulties in Young Children, which specifically outlined 
many of these concerns (Snow et al., 1998, pp. 255-258), the National Reading Panel report, the New 
Zealand Ministry of Education, and various reviews suggesting that such steps would greatly benefit stu- 
dents who are placed in Reading Recovery. 

In summary, the Reading First initiative, recently enacted into law as part of the No Child Left 
Behind Act of 2002, requires the use of scientifically based classroom reading instruction for all students. 
Even with the best classroom instruction, there will still be some students who don’t make adequate 
progress and need additional, more intensive instruction. Reading Recovery has not met the needs of these 
lowest performing students. Most significantly, its excessive costs can make it more difficult for a school 
to provide help for all students in need, especially those who are behind in the upper grades. Thus, 

Reading Recovery is not a productive investment of taxpayers’ money or students’ time and is a classic 
example of a “one size fits all” method. No single method works with all students. Methods like Reading 
Recovery that are rigidly implemented and limited in the number of components of effective reading 
instruction will not work with all students. Reading Recovery leaves too many students behind. 

Sincerely, 



Scott Baker, Ph.D. 

Eugene Research Institute 
University of Oregon 
Eugene, OR 

Virginia W. Berninger, Ph.D. 
Department of Educational 
Psychology 

Research Center on Human 
Development and Disability 
University of Washington 
Seattle, WA 

Maggie Bruck, Ph.D. 

Department of Psychiatry 
Johns Hopkins University 
Baltimore, MD 

James Chapman, Ph.D. 

College of Education 
Massey University 
New Zealand 

Guinevere Eden, Ph.D 
Center for the Study of Learning 
Georgetown University 
Washington, DC 

Batya Elbaum 

Department of Teaching and 
Learning 

University of Miami 
Miami, FL 



Jack M. Fletcher, Ph.D 
Department of Pediatrics 
University of Texas Hlth. Sci. Center 
at Houston 
Houston, TX 

Carol Fowler, Ph.D 
Haskins Laboratories 
New Haven, CT 

David J. Francis, Ph.D. 

Department of Psychology 
University of Houston 
Houston, TX 

Douglas Fuchs, Ph.D. 

Department of Special Education 
Peabody College of Vanderbilt 
University 
Nashville, TN 

Lynn S. Fuchs, Ph.D. 

Department of Special Education 
Peabody College of Vanderbilt 
University 
Nashville, TN 

Keith Greaney, Ph.D 
College of Education 
Massey University 
New Zealand 



Leonard Katz, Ph.D. 

Department of Psychology 
University of Connecticut 
Storrs, CT 

Frank Manis, Ph.D 
Department of Psychology 
University of Southern California 
Los Angeles, CA 

Nancy Mather, Ph.D. 

Department of Education 
University of Arizona 
Tucson, AZ 

Deborah McCutchen, Ph.D. 
Cognitive Studies in Education 
University of Washington 
Seattle, WA 

Einar Mend, Ph.D. 

Department of Pediatrics 
Yale University & Haskins 
Laboratories 
New Haven, CT 

Denise L. Molfese, Ph.D 
Department of Psychology and Brain 
Sciences 

University of Louisville 
Louisville, KY 
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Victoria. Molfese, Ph.D 
Department of Psychology and Brain 
Sciences 

University of Louisville 
Louisville, KY 

Robin Morris, Ph.D. 

Department of Psychology 
Georgia State University 
Atlanta, GA 

Ken Pugh, Ph.D. 

Department of Pediatrics 
Yale University & Haskins 
Laboratories 
New Haven, CT 

Jane Prochnow, Ed.D 
College of Education 
Massey University 
New Zealand 

Christopher Schatschneider, Ph.D. 
Department of Psychology 
University of Houston 
Houston, TX 



Mark Seidenberg, Ph.D. 

Department of Psychology 
University of Wisconsin 
Madison, WI 

Bennett Shaywitz, M.D. 

Department of Pediatrics 
Yale Center for the Study of Learning 
and Attention 
New Haven, CT 

Catherine Snow, Ph.D. 

Harvard Graduate School of 
Education 
Harvard University 
Cambridge, MA 

William Tunmer, Ph.D. 

Department of Learning and 
Teaching 

College of Education 
Massey University 
New Zealand 



Sharon Vaughn, Ph.D. 
Department of Special Education 
University of Texas at Austin 
Austin, TX 

Frank R. Vellutino, Ph.D. 
Department of Psychology 
The University at Albany 
State University of New York 
Albany, NY 

Richard Wagner, Ph.D. 
Department of Psychology 
Florida State University 
Tallahassee, FL 

Maryanne Wolf, Ph. D. 
Department of Psychology 
Tufts University 
Boston, MA 
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UNIVERSITY TRAINING CENTERS 




1 California State University at Fresno 
Dean: Paul Shaker 

Trainers: Jeanette Methven 
Judith Neal 

2 Clemson University 
Dean: Lawrence Allen 
Trainer: Diane DeFord 

3 Emporia State University 
Dean: Tes Mehring 
Trainer: Connie Briggs 

4 Georgia State University 
Dean: Ronald Colarusso 
Trainers: Sue Duncan 

Clifford Johnson 



5 Lesley University 

Dean: William Dandridge 
Trainers: Irene Fountas 
Eva Konstantellou 

6 National-Louis University 
Dean: Elizabeth Hawthorne 
Trainer: Tina Lozano 

7 New York University 
Dean: Ann Marcus 
Trainers: M. Trika Smith-Burke 

Jo Anne LoFaso 
Joe Yukish 
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8 Oakland University 
Dean: Mary Otto 
Trainers: Mary Lose 

Robert Schwartz 
Lee Skandalaris 

9 Purdue University 
Dean: Jerry Peters 
Trainer: Maribeth Schmitt 

10 Saint Marys College 
Dean: Nancy Sorenson 
Trainers: Adria Klein 

Barbara Schubert 

1 1 San Diego State University 
Dean: Lionel Meno 
Trainer: Sharan Gibson 

Patricia Kelly 

12 Shippensburg University 
Dean: Robert Bartos 
Trainer: Janet Bufalino 

13 Texas Womans University 
Dean: Keith Swigger 
Trainers: Nancy Anderson 

Billie Askew 
Betsy Kaye 
Yvonne Rodriguez 

14 The Ohio State University 
Dean: Donna Evans 
Trainers: Mary Fried 

Susan Fullerton 
Carol Lyons 
Gay Su Pinnell 
Emily Rodgers 

15 University of Alabama at Birmingham 
Dean: Michael Froning 

Trainer: Kathleen Martin 



16 University of Arkansas at Little Rock 
Dean: Angela Sewall 

Trainer: Linda Dorn 

17 University of Connecticut 
Dean: Richard Schwab 
Trainer: Mary Anne Doyle 

18 University of Iowa 

Dean: Sandra Bowman Damico 
Trainer: Salli Forbes 

19 University of Kentucky 
Dean: James Cibulka 
Trainer: Judy Embry 

20 University of Maine 
Dean: Robert Cobb 
Trainer: Paula Moore 

21 University of North Carolina- 
Wilmington 

Dean: Cathy Barlow 
Trainer: Noel Jones 

22 University of South Dakota 
Dean: Hank Rubin 
Trainer: Garreth Zalud 

23 Western Michigan University 
Dean: Rollin Douma 
Trainer: Beulah Lateef 
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RESPONSE LETTER 



A BROADER VIEW OF THE EVIDENCE: READING RECOVERY AS AN EXAMPLE 

We are an international group of scholars and researchers who have studied language, literacy, and learn- 
ing in many contexts. We represent a wide variety of perspectives and a range of respected research 
methodologies. 

On the national scene early intervention programs, specifically Reading Recovery, have recently encoun- 
tered one-sided and biased attacks that have misrepresented the efficacy of these programs. We write this 
letter to provide accurate information for the policy decision makers who must protect the interests of chil- 
dren. Of particular concern are children who have extreme difficulty learning to read. Below are five points 
advocating a broad view of the relationships among education, research, and government. 

1. Educational dollars belong to citizens, not to a small group of researchers who have a particular 

point of view. 

Historically, local educational agencies have made decisions, based on their own examination of evi- 
dence, about the programs that will best serve children in their communities. Educational decision 
makers deserve access to a full body of evidence documenting the effectiveness of programs like 
Reading Recovery. 

2. A scientiOc stance requires a complete, evidence-based analysis of any educational program. 

Early intervention has been found to be effective in preventing literacy difficulties.^ For example, for 
more than 20 years, Reading Recovery has been the subject of numerous studies using both quantita- 
tive and qualitative methodology. The preponderance of evidence suggests that this early intervention 
program has a positive and long-lasting effect on reading achievement in young children. Empirically 
controlled studies that are published in high level journals and fit the Department of Education’s crite- 
ria for “scientifically based research” support the results of Reading Recovery.^ Moreover, follow-up 
studies have documented the long-term effectiveness of Reading Recovery early intervention.^ These 
studies have used nationally normed measures. Empirical studies as well as yearly evaluations docu- 
ment the fact that Reading Recovery children grow in self-esteem as they increase their literacy 
skills."^ It is a mistake to deny children access to Reading Recovery based on selective and distorted 
reporting of a few studies, some unpublished or published without peer review, with flawed designs 
and/or with very small populations. 

3. Policy makers have the responsibility to consider evidence from a wide range of perspectives and 

validated research models. 

The attacks on Reading Recovery are based on a very narrow view of evidence that excludes a great 
many high quality and informative quantitative and qualitative studies. A variety of models of research 
have met rigorous criteria in the fields of education, sociology, psychology, anthropology, and other 
areas of social science. We need a range of perspectives in the search for educational improvement. 

For example, qualitative studies provide evidence of Reading Recovery’s profound impact on teacher 
performance and development^ and the dynamic nature of teaching in the program.^ Qualitative 
research, carefully undertaken, represents accepted methodology in a broad number of fields and adds 
significantly to the body of knowledge we must consider in making educational decisions. 

4. Responsibly and rigorously collected evaluation data provide legitimate and strong evidence of 

program success. 

All programs serving children in education should collect and publicly report evaluation data. For 
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example, Reading Recovery’s success has been carefully documented through systematic and simulta- 
neous replications of the program for over one million children in 10,000 schools since its introduc- 
tion into the United States. This documentation provides for public accountability for the progress of 
every^ child sensed. Reading Recovery has consistently reported results of program impact using 
accepted standard measures that are appropriate for young children. These results are available to the 
public.^ When examining any program purporting to be “scientifically based,” policy makers should 
ask for documentation of specific program outcomes for children. 

5. An early intervention program like Reading Recovery is one part of a comprehensive literacy 
effort. 

Early intervention programs serve as safety nets within comprehensive programs and insure that no 
child is left behind. Reading Recovery focuses on the lowest achieving first grade children and works 
in partnership with good classroom instruction, but it is only one component of a comprehensive pro- 
gram. Reading Recovery works as part of many different core instructional models for literacy educa- 
tion including basal approaches. Direct Instruction, Success for All, and other comprehensive pro- 
grams. 

We do not suggest that Reading Recovery or any other program be mandated or given preferential treat- 
ment. We do recommend that the federal government recognize the authority of local teachers, administra- 
tors, and board members to make educational decisions based on full and accurate disclosure of evidence. 

Sincerely, 



Lettie K. Albright, Ph.D. 

Assistant Professor 
Department of Reading 
Texas Woman’s University 

JoBeth Allen 
Professor 

Language Education 
University of Georgia 

Richard L. Allington, Ph.D. 

Irving and Rose Fien Distinguished 
Professor of Elementary and 
Special Education 
School of Teaching and Learning 
University of Florida 

Mark Alter, Ph.D. 

Chair and Professor 
Teaching and Learning 
New York University 

Patricia L. Anders, Ph.D. 

Professor 

Department of Language, Reading, & 
Culture 

University of Arizona 

Terry A. Astuto, Ed.D. 

Professor of Educational 
Administration and Department 
Chair 

Administration, Leadership and 
Technology 
New York University 



Kathryn Au, Ph.D. 

Dai Ho Chun Professor of Education 
Teacher Education & Curriculum 
Studies 

University of Hawaii 

Mary Kathleen Barnes, Ph.D. 
Assistant Professor 
School of Teaching and Learning 
The Ohio State University at Marion 

Constance K. Barsky, Ph.D. 

Director 

Learning by Redesign 
The Ohio State University 

Eurydice Bauer, Ph.D. 

Assistant Professor 
University of Illinois at Urbana- 
Champaign 

Penny Beed, Ph.D. 

Associate Professor and Coordinator 
of Literacy Education 
Curriculum and Instruction 
University of Northern Iowa 

Mary Bendixen-Noe, Ph.D. 

Associate Professor 

School of Teaching & Learning 

The Ohio State University 

Jacques S. Benninga, Ph.D. 

Director 

Bonner Center for Character 
Education 

California State University at Fresno 



Laura Benson 

Literacy Consultant, Writer & 
College Instructor 
Literacy and Language 
University of Colorado at Denver 

Mollie Blackburn, Ph.D. 

Assistant Professor 

School of Teaching and Learning 

The Ohio State University 

David Booth, Ph.D. 

Professor 

Curriculum, Teaching and Learning 
University of Toronto 

Gregory W. Brooks, Ph.D. 

Assistant Professor 
Education Department 
Nazareth College of Rochester 

Anthony S. Bryk, Ed.D. 

Marshall Field IV Professor of Urban 
Education and Sociology 
Director of the Center for School 
Improvement and the Consortium 
on Chicago School Research 
University of Chicago 

Terry L. Bullock, Ed.D. 

Associate Professor 
Reading and Critical Thinking 
University of Cincinnati 
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Marsha Riddle Buly, Ph.D. 

Assistant Professor 
Elementary Education 
Western Washington University 

Judith Anne Calhoon, Ph.D. 

Assistant Professor 
Teaching and Leadership 
University of Kansas 

Lucy McCormick Calkins, Ph.D. 
Professor of English Education 
Columbia University 
Teachers College 

Thomas A, Caron, Ph.D. 

Professor 

Reading Education 

Marshall University Graduate College 

Kathryn S. Carr, Ed.D. 

Professor Emerita 
Department of Curriculum and 
Instruction 

Central Missouri State University 

Courtney B. Cazden, Ed.D. 

Charles William Eliot Professor of 
Education (Emerita) 

Harvard Graduate School of 
Education 
Harvard University 

Caroline T Clark, Ph.D. 

Associate Professor 
Language, Literacy & Culture 
The Ohio State University 

Thomas Cloer, Jr, Ph.D. 

Professor of Education 
Department of Education 
Furman University 

Sheila G. Cohen, Ed.D. 

Associate Professor 
Literacy Department 
SUNY Cortland 

Margaret Compton-Hall, Ed.D, 
Assistant Professor 
Department of Reading 
Texas Woman’s University 

Van Cooley, Ed.D. 

Professor and Chair 

Teaching, Learning, and Leadership 

Western Michigan University 

Beverly E. Cox, Ph.D. 

Associate Professor of Literacy and 
Language 

Curriculum and Instruction 
Purdue University 



Ronald L. Cramer, Ph.D. 
Distinguished Professor of Education 
Reading & Language Arts 
Oakland University 

Ronald Crowell, Ph.D. 

Professor of Education 
Teaching, Learning, and Leadership 
Western Michigan University 

Bernice Cullinan, Ph.D. 

Professor Emerita 
Department of Teaching and 
Learning 

New York University 

James W. Cunningham, Ph.D. 
Professor of Literacy Education 
School of Education 
UNC - Chapel Hill 

Patricia M. Cunningham, Ph.D. 
Professor 

Department of Education 
Wake Forest University 

Karin Dahl, Ph.D. 

Professor 

School of Teaching and Learning 
The Ohio State University 

Sandra Bowman Damico, Ph.D. 

Dean 

College of Education 
University of Iowa 

William L. Dandridge, Ed.D. 

Dean 

School of Education 
Lesley University 

Sheryl Dasinger 
Assistant Professor 
Early Childhood and Reading 
Valdosta State University 

Pamela Dougherty-Smith, Ph.D. 

Lead Reading Teacher 

Dallas Independent School District 

Ann M. Duffy, Ph.D. 

Assistant Professor 
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Sarah Edwards, Ph.D. 

Assistant Professor 
Teacher Education 
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